Nanopass Roadmap

This document proposes a nanopass-inspired compiler architecture for RukaLang. It is a migration plan, not an all-at-once rewrite.

Assumptions

• The main priorities are maintainability and clarity of invariants.
• Small compile-time regressions are acceptable if we gain much better architecture hygiene.
• We keep the current top-level stage order (syntax -> meta -> elab -> hir -> check -> mir -> codegen) and split inside stages first.
• We preserve current language behavior while we refactor pass structure.

If these assumptions change, we should revise this plan before implementation.

Assumptions confirmed by maintainer (2026-03-18).

Design Goals

1. Minimize boilerplate when adding passes and intermediate forms, using macros or other metaprogramming constructs where appropriate.
2. Carry source information through the whole pipeline with one shared representation.
3. Maximize allocation performance with arena-based storage (cranelift_entity first choice).
4. Make pass contracts explicit: each pass declares required input invariants and guaranteed output invariants.
5. Framework should be split into its own crate to allow potential separate publication & reuse in other compilers later, but supporting RukaLang should be the top priority.

Core Architecture

1) Pass Interface With Low Boilerplate

Introduce a shared pass API with a small descriptor and one run method.

pub trait Pass {
    type In;
    type Out;
    type Error;

    const NAME: &'static str;

    fn run(&mut self, input: Self::In, cx: &mut PassContext) -> Result<Self::Out, Self::Error>;
}

PassContext carries shared facilities:

• interners,
• source/provenance tables,
• diagnostics sink,
• reusable scratch arenas,
• stats/timing hooks.

To reduce repeated code, add a small helper macro for pass declaration metadata and a pass runner that wires logging/timing/diagnostic framing once.

2) Shared Source + Provenance Representation

Use one representation for all source and expansion provenance:

• SourceFileId (entity_impl! newtype)
• SpanId pointing into a global span arena
• OriginId for synthetic/generated nodes

Each IR node stores either:

• a direct compact SpanId, or
• an OriginId when generated from other nodes.

OriginId resolves through a provenance graph:

• Origin::Parsed { span }
• Origin::Expanded { from: OriginId, phase: PassId }
• Origin::Lowered { from: OriginId, phase: PassId }
• Origin::Synthesized { reason, parent: Option<OriginId> }

This gives one diagnostics path from any late IR entity back to user source, including meta expansion and lowering.

3) Allocation Model

Use PrimaryMap for arena-owned entities and SecondaryMap/side tables for annotations:

• PrimaryMap<EntityId, Node> for nodes,
• SecondaryMap<EntityId, SpanId> for direct source,
• SecondaryMap<EntityId, OriginId> for provenance,
• compact index-backed vectors for analysis facts.

Guidelines:

• pre-size maps from cheap counts when possible,
• avoid cloning large subtrees across passes; prefer id remapping tables,
• keep per-pass temporary state in scratch arenas owned by PassContext, then clear/reuse.

Pipeline Refactor Plan

Current implementation reference:

• See Pass Inventory for the current typed pass list, execution order, and implementation links.

Implementation Status (2026-03-19)

Completed so far:

1. Pass framework landed in src/pass with typed pass execution, pass ids, timing capture, and shared provenance tables (SourceFileId, SpanId, OriginId).
2. Top-level production pipeline now runs through typed pass wrappers for all current stages: meta, elab, hir, check, mir, codegen.rust, codegen.wasm.
3. Elaboration split-in-progress: core runtime call/template concerns are now explicit subpasses:
   • elab.normalize_runtime_calls_and_spreads
   • elab.validate_runtime_call_args
   • elab.bind_template_call_args
   • elab.instantiate_runtime_function
4. Per-pass observability landed:
   • pass timings (--dump-pass-timings)
   • pass snapshots (--dump-pass-snapshots)
   • JSONL snapshots with schema/version (--dump-pass-snapshots-json)
5. Provenance side-table implementation started:
   • HIR expression origin side tables
   • MIR local origin side tables
   • origin chains include Parsed -> Expanded(meta) -> Lowered(elab) -> Lowered(hir[/mir])
6. Browser/WASM analysis path migrated onto driver-based pipeline hooks.
7. CI now includes a browser WASM API smoke check to catch runtime regressions in compile/analyze behavior.

Remaining major work:

1. Continue decomposition of elab until major mixed-responsibility blocks are isolated behind stable pass contracts.
2. Start check phase split (collect_decls, resolve_signatures, etc.).
3. Extend provenance mapping to more node/entity kinds and tighten diagnostic source reconstruction quality.
4. Add stronger fixture/snapshot coverage for pass contracts and structured snapshot schema stability.

Phase 0: Infrastructure First

Deliverables:

1. pass crate/module with Pass, PassContext, PassId, pass runner.
2. Shared provenance tables and IDs (SourceFileId, SpanId, OriginId).
3. Compiler driver updates to run a pass list and emit pass timing stats.

Exit criteria:

• current behavior unchanged,
• existing tests pass,
• source spans still appear in diagnostics.

Status:

• Complete.

Phase 1: Split elab Into Explicit Subpasses

Current elab mixes many concerns. First split candidate:

1. collect_runtime_templates
2. resolve_type_names
3. instantiate_runtime_templates
4. infer_runtime_expr_types
5. normalize_runtime_calls_and_spreads
6. runtime_type_validation

Each subpass operates over one arena-backed runtime AST form with side tables, not deep cloning.

Exit criteria:

• golden tests for per-pass output snapshots,
• invariants documented for each pass,
• no language behavior drift.

Status:

• In progress. Core runtime call/template concerns now run as explicit elab subpasses (normalize_runtime_calls_and_spreads, validate_runtime_call_args, bind_template_call_args, instantiate_runtime_function).

Phase 2: Split check Into Independent Analyses

Suggested decomposition:

1. collect_decls
2. resolve_signatures
3. check_expr_and_stmt_types
4. check_loans_and_moves
5. finalize_checked_program

Store analysis outputs in compact side tables keyed by expression/statement ids.

Exit criteria:

• diagnostics parity for current fixtures,
• checker internals no longer require one giant mutable state object.

Phase 3: Split mir_lower

Suggested decomposition:

1. build_function_skeletons
2. lower_cfg
3. lower_types_and_layout
4. insert_runtime_intrinsics
5. mir_sanity_validation

Exit criteria:

• MIR graph parity on fixture corpus,
• no codegen regressions in Rust/WASM outputs.

Phase 4: Optional Full Nanopass Expansion

After phases 1-3, we can choose finer granularity pass-by-pass.

Decision gate:

• if a pass still has mixed responsibilities or weak invariants, split again,
• if not, keep current granularity.

This keeps a path to full nanopass architecture without forcing every split immediately.

Boilerplate Reduction Strategy

1. Use generated EntityId newtypes (entity_impl!) and common arena wrappers.
2. Keep one pass registration table:
   • pass name,
   • input/output type ids,
   • optional debug dump hook.
3. Auto-wire pass logging, timing, and panic context in one runner.
4. Reuse traversal helpers for common AST/HIR/MIR walk patterns.

Source/Diagnostics Strategy

1. Every emitted diagnostic must carry an OriginId.
2. Diagnostics rendering resolves origin chain to best user-facing span.
3. If multiple source candidates exist (for generated nodes), render:
   • primary span,
   • one secondary note with expansion/lowering origin.

This keeps diagnostics robust as pass count grows.

Performance Strategy

1. Prefer arena ids over owned recursive trees in inner passes.
2. Keep hot tables in flat vectors keyed by entity index.
3. Batch allocate nodes and annotations per pass; avoid per-node heap allocations.
4. Collect and track pass timing/allocation counters from day one of migration.

Validation Plan

At each phase:

1. Run cargo test.
2. Run ./scripts/ci.sh before PR.
3. Add fixture tests for any new diagnostics surface.
4. Add pass contract tests:
   • checks for required input invariants,
   • checks for guaranteed output invariants.

Decisions (Confirmed)

1. Pass errors use per-pass error enums wrapped by one top-level compiler error type.
2. Provenance uses one canonical OriginId path; parsed nodes are Origin::Parsed { span }.
   • Storage policy: keep OriginId in side tables keyed by arena entity ids, not as direct IR node fields.
   • Rationale: lower node-size overhead, less constructor/pattern-match churn, one shared provenance representation.
3. Subpasses prefer in-place mutation over arena-backed IR + side tables, and emit a new IR only when structure must change.
4. Pass registration starts as a static compile-time pass list (typed, no dynamic dispatch).
5. Expose pass-level debug dumps through CLI flags in phase 0.
6. Persist provenance graph in browser artifacts and revisit graph presentation as pass count grows.
7. Use one shared IR node id namespace per stage (not per-module) for maintainability.

Suggested First Implementation Slice

Keep this first slice small and reversible:

1. Add PassContext + provenance ids/tables.
2. Wrap existing elab::elaborate_program as a single pass under new runner.
3. Split only one elab concern (normalize_runtime_calls_and_spreads) into its own pass.
4. Verify diagnostics parity and benchmark compile time on fixture corpus.

If this slice lands cleanly, continue with the rest of phase 1.