Compiler

koskari.compile

The koskari compiler. Converts source forms into bytecode fragments.

class koskari.compile.ByteCodeFragment(compiler=None, name=None, filename=None, resolution=None)

Bases: object

Holds a fragment of bytecode during compilation: instructions plus closure requests and name declarations for the namespace

as_expression(parent)

as_function(parent, formals_spec=((), ()), defaults=None)

as_function_spec(formals_spec=((), ()))

as_namespace_spec()

Build a NamespaceSpec from this fragment’s name and declarations.

child_namespace(ns_name, names, values)

Context manager that compiles a child scope with bound names. On entry, compiles all value expressions and yields a fragment for the body. On exit, emits value expressions (if any), then PUSH_NS with nspec and names in the EXTRA row, body instructions, POP_NS.

Parameters:

• ns_name – name for the child namespace (e.g. source form)

• names – sequence of bound names, symbols

• values – sequence of expressions (one per name)

collector_namespace(ns_name)

Context manager for a collector namespace (CollectorNamespace). Like child_namespace but uses PUSH_COLLECT/POP_COLLECT. POP_COLLECT pushes the collected namespace onto the stack.

Parameters:

• ns_name – name for the child namespace

• names – sequence of bound names (strings or symbols)

• values – sequence of expressions (one per name)

declare_var(var_name, position=None, type_expr=None)

Record a name declared in this fragment’s namespace.

Parameters:

• var_name – name to declare

• position – optional (line, col) for error reporting

• type_expr – optional source form for a type annotation (compile metadata only; not evaluated here)

declare_vars(var_names)

Record multiple names declared in this fragment’s namespace

Parameters:

var_names – iterable of names to declare

instruction(op, opa)

Append one bytecode instruction

Parameters:

• op – OpCode for the instruction

• opa – argument for the instruction; a variable operand, the opcode’s own or a variable field of a fused row, is a symbol

Raises:

TypeError – when a variable operand is not a symbol

marked = False

merge(other)

Merge another fragment into this one: adopt its declarations, requests, requests_for_write, request_global, and annotation_bindings, then append its instructions.

Used when combining forms in the same scope (e.g. begin body); request_global must propagate so that (define-global x) followed by (setq x 1) in the same block works.

note_mark(sym)

Record that this fragment references a marked name, so its completion resolves its names

Parameters:

sym – a gensym

position(linecol, extra=None)

Emit a position instruction

Parameters:

• linecol – (line, column) or position tuple

• extra – optional extra position data (e.g. source text)

position_of(source)

Emit a position instruction from the source’s position if set

Parameters:

source – a pair or object with get_position

request_for_write(var_name, position=None)

Record that var_name must be writable (setq, delq, inplace).

Parameters:

• var_name – name that must be writable

• position – optional (line, col) for error reporting

request_global(var_name, position=None)

Record explicit need from module (define-global).

Parameters:

• var_name – name to pull from module

• position – optional (line, col) for error reporting

request_var(var_name, position=None)

Record a closure request for var_name unless it is already declared

Parameters:

• var_name – name to request from enclosing scope

• position – optional (line, col) for error reporting

rewrite_vars(choose)

Replace each variable operand in the instruction stream with choose(name, scopes), scopes holding the spelling sets of the inline scopes open around it, innermost first. An inline scope’s closure names are chosen from outside that scope

Parameters:

choose – callable taking an atom and the open scopes, returning the atom to emit

class koskari.compile.Compiler(name, module, filename=None, tco_enabled=True, resolution_mode=ResolutionMode.KOSKARI, bracket_calls=BracketCalls.IGNORE, stray_colon=StrayColon.ALLOW, stray_hash=StrayHash.ALLOW, keyword_literals=KeywordLiterals.ALLOW)

Bases: object

property bracket_calls

check_keyword_literal(kwd, pos=None)

Diagnose an unquoted keyword compiled as a literal value per the keyword_literals policy. It evaluates to itself, which is legal but is far more often a typo – (if foo: 1 2) tests the keyword foo: for truthiness (always true) instead of reading foo.

Quoting is exempt: the quote special emits its own constant and never reaches here. Keyword arguments are exempt too, being consumed structurally by gather_parameters rather than as values.

Keywords are interned and so carry no position of their own; pos is the enclosing form’s (line, col), used to attribute a WARN. It defaults to the innermost form compiled so far.

Parameters:

• kwd – the keyword about to be compiled as a constant

• pos – position of the enclosing form, or None

check_rewrite_cycle(seen, form)

Guard the full_compile rewrite loop: a rewrite that repeats an earlier form in this position is a macro-expansion cycle, and a chain past the settling limit is divergent growth; either way the compile stops here instead of spinning.

Identity is exempt from the cycle check: a special may mutate its source in place and return the same object to request a recompile (the return/yield keyword-values fixup), which is progress, not a cycle. A genuinely stuck identical form falls to the settling limit instead.

The settling limit comes from the compiler/rewrite-limit pragma setting; 0 disables the limit while the cycle check stays in force.

Parameters:

• seen – prior rewrite forms in this position, oldest first

• form – the newly produced rewrite

Raises:

CompilerError – on a cycle or an unsettled chain

check_stray_symbol(sym, pos=None)

Diagnose a bare : or # symbol compiled as a name per the stray_colon or stray_hash policy. Nothing binds either, so reaching here is always a mistake. A stray : is usually a layout form half-converted to s-expressions, where the trailing block marker of (for-each [x (xs)]: body) reads as a symbol element rather than as a block colon. A stray # is usually a typo for a collection opener or a syntax quote.

Symbols are interned and so carry no position of their own; pos is the enclosing form’s (line, col), used to attribute a WARN. It defaults to the innermost form compiled so far.

Parameters:

• sym – the symbol about to be compiled as a name

• pos – position of the enclosing form, or None

compile(source_obj, *, position=None)

compile_const(fragment, value, position=None)

compile_expr(fragment, source_obj, position=None)

position is the innermost enclosing source position known to the caller, threaded to diagnostics about positionless atoms; a source object carrying its own position wins over it

compile_expression(source_obj, position=None)

Compile one top-level source expression: the entry point for the places a compilation turn starts – the module loaders, the REPL, the serializer, and any harness whose pragma should govern the turn. Binds this compiler’s pragma into the active-compiler-pragma parameter for the turn’s extent, so macro bodies running during the compile read the compiling module’s pragma through (active-pragma), regardless of which module defined them. Inner recursive compilation chains through compile and full_compile directly and never rebinds.

Parameters:

• source_obj – top-level source form

• position – where the reader found the form, for a form that carries no position of its own (an atom)

Returns:

compiled ByteCodeFragment

compile_pair(fragment, source_obj, position=None)

compile_raw_symbol(fragment, sym, position=None)

compile_special(fragment, comp, source_obj)

compile_symbol(fragment, sym, position=None)

complete(fragment, mode=None, _validate=True, is_root=False, ancestors=None)

Complete a fragment into a Code object.

The Deferred constants the fragment holds complete first, against this fragment’s now final declarations, and their values take their place

Parameters:

• fragment – the fragment to complete

• mode – the mode to complete in

• _validate – whether to validate the code

• is_root – whether this is the root fragment

• ancestors – the fragments lexically enclosing this one, outermost first; the open lexical stack when omitted

Returns:

the completed Code object

complete_function(fragment, ancestors)

Complete a function body fragment under the given ancestry

Parameters:

• fragment – the function body fragment

• ancestors – the fragments enclosing it, outermost first

Returns:

the completed Code object

find_bare_compiled(name)

Resolve as find_compiled(), keeping only a Compiled that declares a bare form. Anything else resolves as not found, so a symbol naming an ordinary special still compiles as a variable; prefixed spellings never resolve as uses, per find_compiled

Parameters:

name – symbol or string for the compile-time object

Returns:

Compiled instance with a bare form, or None

find_compiled(name)

Resolve compile/<name> in the module namespace

Parameters:

name – symbol or string for the compile-time object (e.g. if, begin)

Returns:

Compiled instance, or None

fragment(name=None)

classmethod from_pragma(name, module, filename=None, pragma=None)

Create a compiler from a pragma, or a default pragma and its settings.

full_compile(source_obj, *, position=None)

property keyword_literals

limit_error(fragment, ex)

A size the processor or the finalizer refused, reported as a compiler error at the fragment’s first recorded position

lower_head(form, position=None)

Attempt to lower a form found in operator position. Caller: compile_pair.

The attempt nests through arbitrarily deep pair heads: each level resolves its own head, and only a form whose innermost head names a syntax is attempted at all. A syntax whose expansion yields a source form succeeds, reconstructing each enclosing level around the result; one that compiles to bytecode answers with that fragment, which the caller remembers against the head so the call path’s compile of the same head is that fragment and the head is never compiled twice. A non-syntax head fails the whole attempt, committing nothing.

Parameters:

• form – the pair found in operator position

• position – enclosing source position threaded to diagnostics

Returns:

the lowered replacement form, a fragment the head compiled to, or None when the attempt fails

macro_overlay(name, syntax)

macro_overlays(**dfns)

Context manager that overlays the macro namespace with the given definitions.

open_allocator()

Open a CodeAllocator for a module load, unless one is already open. Function code completed from here until the next seal is placed into its arena chunks while compiler/pack-module is on

Returns:

the open CodeAllocator

position_atom(fragment, position)

An atom carries no position; when it is the whole of what the fragment holds so far, the position threaded from the reader names its line

reload_pragma(pragma)

Reload the compiler from a pragma.

property resolution_mode

resolution_scope(fragment=None, function=False)

Bracket the compilation of a nested scope’s body

Parameters:

• fragment – the fragment the body compiles into; while the body is open it sits on the lexical stack as an enclosing scope of anything completed inside it

• function – the fragment is a function body, the kind of scope a Code is built from

scope_overlay(name, syntax)

Overlay the macro namespace with one definition until the innermost open resolution scope, or the enclosing macro_overlay extent, closes

Parameters:

• name – the binding spelling, compile/NAME

• syntax – the Compiled object to resolve it to

seal()

Seal the arena chunks this module load placed function code into, read-only, and close the load’s allocator. Code completed afterwards is a heap block until a later load opens another.

Idempotent: with no allocator open, there is nothing to seal. Nothing moves, so sealing is safe at any point between two completions.

Returns:

tuple of the sealed Arena chunks; empty when no allocator was open or nothing was placed

property stray_colon

property stray_hash

symbol_is_variable(sym)

Whether a lone symbol compiles to a variable read, so a special may name it in a fused VAR operand: it is not a compiler literal and no bare arm expands it. Anything else must go through full_compile, which expands the bare arm or emits the constant

Parameters:

sym – the symbol as written, marks included

exception koskari.compile.CompilerError(message, source=None, position=None, filename=None)

Bases: KoskariException

Base class for compile-phase errors

attach_position(position=None, filename=None)

Fill in a position and a filename the raise site did not have: the innermost enclosing form’s position and the compiler’s file. What the error already carries is kept

explain()

User-facing error text for REPL and similar UIs

class koskari.compile.CompilerMode(*values)

Bases: IntEnum

EXPRESSION = 2

FUNCTION = 1

exception koskari.compile.ParametersError(message, source=None, position=None, filename=None)

Bases: CompilerError

Call-site parameter errors from koskari.compile.helpers.gather_parameters() validation

exception koskari.compile.SpecialError(message, source=None, position=None, filename=None)

Bases: CompilerError

Special-form validation errors

exception koskari.compile.SyntaxMatchError(message, source=None, position=None, filename=None, *, syntax=None, patterns=None)

Bases: SpecialError

Syntax dispatch errors when no matcher arm matches the source form

explain()

User-facing error text for REPL and similar UIs

exception koskari.compile.UncaughtCompilerError(message, source=None, position=None, filename=None)

Bases: CompilerError

Wraps unexpected exceptions raised during compilation