diff --git a/docs/superpowers/specs/2026-09-13-libvct-design.md b/docs/superpowers/specs/2026-09-13-libvct-design.md new file mode 100644 index 0000000..7bc6064 --- /dev/null +++ b/docs/superpowers/specs/2026-09-13-libvct-design.md @@ -0,0 +1,788 @@ +# libVCT — Vox C Transpiler Library: Architecture Specification + +- **Status:** Draft (pending final review) +- **Date:** 2026-09-13 +- **Scope:** Whole-library architecture — component boundaries and interfaces. Subsystem internals are deferred to later specs. +- **Normative language:** `MUST`, `SHOULD`, `MAY` per RFC 2119. + +--- + +## 1. Overview + +libVCT is an LLVM-inspired compiler library written in D. It consumes an IR, optimizes it, and transpiles it into C17, then drives a system C compiler to produce an object file. An alternate backend emits textual LLVM IR instead of C. + +The library is organized around a **two-stage IR**: + +- **HIR** — a high-level, tree-shaped AST that is the frontend's output and is **annotated in place** with *traits*. HIR does discovery and small, cheap folds. +- **VIR** — a low-level SSA/CFG instruction IR. VIR does the heavy lifting and performs **no discovery**; it is a strict consumer of HIR-provided traits. + +The design thesis: because HIR hands VIR a fully annotated "dictionary" of facts, VIR can skip analysis and go straight to transformation. HIR's cheap, cascading folds are parallelizable; the resulting VIR is simpler and faster to optimize. The net goal is LLVM-class codegen quality with materially lower compile time. + +### 1.1 Goals + +1. **Codegen quality is the headline.** Generated code quality takes priority over compile speed. +2. **Two-stage optimization.** HIR performs small folds and trait derivation; VIR performs large transformations. +3. **Trait-driven VIR.** VIR never re-analyzes; every fact it uses is an attribute, relationship, or request supplied by HIR. +4. **Parallel, cascading HIR.** HIR passes run per-function in parallel and cascade to a fixpoint. +5. **Compiler-friendly C.** The C backend always emits C written to be pattern-matched by GCC/Clang. +6. **Deterministic output.** Same input + same config ⇒ byte-identical output, regardless of thread count. + +### 1.2 Non-goals (v1) + +- **Linking.** libVCT produces object files; linking is the caller's job. The sole exception is `-mangled` (cosmopolitan) mode. +- **Windows / MSVC.** Linux/POSIX-first; gcc and clang only. +- **LLVM bitcode.** The LLVM backend emits textual `.ll` only; the caller runs `llvm-as`. +- **LLVM debug metadata.** No `!llvm.dbg` in v1; debug mapping exists on the C path via `#line`. +- **A user-facing VIR.** VIR is internal and intentionally hostile to hand-authoring. +- **Consuming LLVM IR or Vox's existing IR.** libVCT owns its IR definition. + +### 1.3 Glossary + +| Term | Meaning | +|---|---| +| **HIR** | High-level IR: the frontend AST, annotated in place with traits. | +| **VIR** | Vox IR: a low-level SSA/CFG instruction IR. | +| **Trait** | The union of Attribute, Request, and Relationship attached to a node. | +| **Attribute** | A fact about a node (type, value, mutability, ...). | +| **Request** | An attribute-derived suggestion from HIR to VIR. | +| **Relationship** | A directed edge between nodes, traceable up/down. | +| **Cascade** | The chain of HIR rewrites triggered by one fold. | +| **Lowering** | The "middleman" that translates annotated HIR into VIR SSA. | +| **Remark** | An optimizer decision record emitted for diagnostics. | + +--- + +## 2. Component Architecture + +### 2.1 Modules + +| Module | Responsibility | Allocation | Visibility | +|---|---|---|---| +| `vct.ir.hir` | HIR AST node types, Trait model, builder/emitter API | arena | **public** | +| `vct.ir.vir` | VIR SSA/CFG types, VIR type system, instruction set | arena | internal | +| `vct.traits` | Attribute / Request / Relationship definitions + queries | arena | **public** | +| `vct.comptime` | Comptime evaluator / constant-folding engine | arena | internal | +| `vct.hir.opt` | HIR pass framework + passes, parallel cascade | arena | internal | +| `vct.lower` | HIR → VIR lowering + SSA construction | arena | internal | +| `vct.vir.opt` | VIR pass framework + heavy passes | arena | internal | +| `vct.backend.c` | VIR → C17 emitter (out-of-SSA, compiler-friendly C) | arena + GC | **public entry** | +| `vct.backend.llvm` | VIR → textual LLVM IR emitter | arena + GC | **public entry** | +| `vct.driver` | cc invocation, flag assembly, `.o` emission, LTO, cosmopolitan | GC | **public** | +| `vct.cli` | Thin command-line wrapper | GC | **public** | +| `vct.diag` | Diagnostics, source maps, error reporting | GC | **public** | +| `vct.context` | Context / arena ownership, `Config` | GC + arena | **public** | +| `vct.test.hirbuild` | Fluent HIR test-builder harness | arena | **public** | + +### 2.2 Data flow + +``` +frontend --build--> HIR + --hir.opt (parallel, cascading)--> annotated HIR + --lower--> VIR (SSA/CFG) + --vir.opt--> optimized VIR + --backend.c--> C17 source --driver--> .o (default path) + +optimized VIR --backend.llvm--> textual LLVM IR (caller links LLVM) +``` + +### 2.3 Public interfaces + +1. **HIR builder (frontend-facing).** Construct nodes, set types/values, attach frontend-supplied traits (`static`, hints), and finish a module. +2. **Trait query (VIR-facing).** Read attributes and walk relationships. This is the **only** channel through which VIR obtains facts. +3. **Backend (driver-facing).** Consume optimized VIR and emit C (or textual LLVM IR). + +### 2.4 Load-bearing boundaries + +- **`vct.traits` is the contract.** Both HIR and VIR depend on it. VIR depends on nothing else from HIR except **node identity**. +- **`vct.lower` is the SSA construction site.** Phi-nodes, dominance, and the memory model all land here; it is the most algorithmically dense module. + +--- + +## 3. The Trait Model + +A `Trait` is the union of three sub-structures hung on HIR nodes, and the entire vocabulary VIR may reason with: + +```d +struct Trait { + Attribute attr; + Request[] reqs; + Relation[] rels; +} +``` + +Traits are **exposed publicly**. A frontend MAY **suggest** traits to HIR; HIR validates suggestions and fills in correct values. Derived traits are authoritative and VIR trusts them blindly. + +### 3.1 Attributes + +Attributes are facts about a node, with **two origins in one namespace**: + +- **Suggested** — set by the frontend at build time. HIR **validates**; on mismatch it emits a warning (promoted to error under `-Werror`) and **overwrites** with the correct value. +- **Derived** — produced by the HIR optimizer during the cascade. Authoritative. + +Every attribute carries an `attr_source` bit recording whether it was suggested or derived. + +**v1 attribute set:** + +| Attribute | Meaning | +|---|---| +| `ty` | The node's VIR type. | +| `const_value` | Present iff the value is known at compile time. | +| `is_static` | Frontend-declared compile-time value. | +| `is_comptime` | HIR-proven compile-time value. | +| `is_constant` | Value is constant (not necessarily comptime). | +| `is_used` | Node has at least one use. | +| `is_mutably_used` | Node is mutated through at least one use. | +| `is_addressed` | Address taken by pointer/reference. | +| `escapes` | Value escapes its defining scope. | +| `is_runtime_mutable` | May change at runtime. | +| `may_change_at_runtime` | Alias of intent: value is not frozen. | +| `complex` | Frontend-provided; gates the `NoOptimize` request. | + +### 3.2 Requests + +Requests are **attribute-derived** suggestions from HIR to VIR — never ad-hoc. Each has a **strength**: + +- `Soft` — a suggestion VIR MAY decline. +- `Strong` — violating it is likely a bug (e.g., `NoInline`, `MustTail`, `AlwaysInline`, `NoOptimize`). + +Every request produces a `RequestResult`: + +```d +struct RequestResult { + Request req; + bool accepted; + DenyReason reason; + string note; // one-line human-readable explanation +} +``` + +A denial MUST carry a `DenyReason` enum (for tooling) and a one-line `note`. + +```d +enum DenyReason { + CostModel, // profitable only under a different cost model + Illegality, // transform would be incorrect + AlreadyDone, // no-op: the goal already holds + Unsupported, // VIR does not implement it yet + ContradictsTrait, // conflicts with a stronger fact + TooLarge, // exceeds a size budget + NoTargetSupport, // backend/target cannot express it +} +``` + +#### 3.2.1 Request catalog (v1) + +**Inlining / call edges:** `Inline`; `AlwaysInline` (Strong); `NoInline` (Strong); `TailCall`; `MustTail` (Strong); `NoTail` (Strong); `Devirtualize`; `ColdCall`; `LikelyCall`; `UnlikelyCall`. + +**Loops:** `Vectorize`; `Unroll(factor)`; `Interleave(count)`; `Peel(count)`; `Distribute`; `Fuse`; `Jam`; `Unswitch`; `Rotate`; `LICM` (action); `LoopInvariant` (property); `MustProgress`. + +**Memory effects:** `NoAlias`; `Restrict`; `NonNull`; `Align(n)`; `ReadOnly`; `WriteOnly`; `NoRead`; `NoWrite`; `NoCapture`; `Dereferenceable(n)`; `Constant`; `NoUndef` (reserved); `Prefetch`. + +**Control flow:** `HotPath`; `ColdPath`; `Unreachable`; `NoReturn`. + +**Assumptions:** `Assume(pred)`; `Range(lo, hi)`. + +**Optimization control:** `NoOptimize` (Strong; requested **only** when the frontend sets `complex` on the node). + +#### 3.2.2 Request conflict rules + +- `Strong` beats `Soft`. +- Two conflicting `Strong` requests (e.g., `Inline` vs `NoInline`, `TailCall` vs `NoTail`) ⇒ HIR emits a diagnostic and **rejects** the frontend suggestion. +- `LoopInvariant` (a proven property VIR may exploit) and `LICM` (an explicit request to hoist) are **distinct** and both retained. +- `NoRead`/`NoWrite` are **primitives on an access edge**; `ReadOnly`/`WriteOnly` are **region-level facts** derived from them. All four are retained, with the scope difference documented. + +### 3.3 Relationships + +Relationships are directed edges HIR leaves for the non-constant world. They are traceable **up and down** to a terminal node. + +**Flags:** + +| Flag | Meaning | +|---|---| +| `is_offspring` | Cannot trace up. | +| `is_ancestor` | Cannot trace down. | +| `is_common` | Branch point; VIR must choose a direction. | +| `has_siblings` | Connects down to 2+ nodes. | +| `is_apex` | Part of a cycle (`a → b → c → d → a`), usually from polymorphism. | + +**Relation attributes (small set):** `was_changed`; `is_mutable`; `is_pointer`. + +An apex SHOULD normally be ironed out by HIR; residual apexes typically arise from polymorphism. + +### 3.4 Representation & staleness + +- **Representation (resolved):** an **embedded dense `Trait` struct** on every node is primary, plus an **optional sparse side-channel** for rare / frontend-extensible attributes. +- **Staleness (resolved):** **epoch/generation counters** with dirty propagation along relationships. A trait is valid only for the node's current epoch. This survives parallel HIR passes. + +--- + +## 4. HIR Optimizer + +**Role:** discovery + small folds + trait derivation. HIR hands VIR a "dictionary". + +### 4.1 Pass framework + +```d +interface HirPass { + string name(); + void run(HirFunction fn, HirContext ctx); +} +``` + +Passes mutate the annotated AST in place. Successful rewrites emit **triggers**: + +```d +enum Trigger { + ConstantUnfolded, + UsesReplaced, + NodeDeleted, + TraitChanged, + StaticDiscovered, +} +``` + +A **worklist scheduler** collects `(pass, node)` pairs, dedupes, and runs to a fixpoint. A per-function step budget guarantees termination. + +### 4.2 The cascade + +Canonical chain: + +``` +4 * 16 + -> const 64 + -> mark is_comptime / is_constant / const_value = 64 + -> prove not addressed / not runtime-mutable + -> replace all uses with literal 64 + -> node dead -> delete + -> downstream nodes now constant -> enqueue +``` + +Each hop emits a trigger; the worklist drains until nothing new is provable. **HIR never inlines** — it only proves smallness and issues an `Inline` request to VIR. + +### 4.3 Parallelism & determinism + +- Parallelize **per function** (independent HIR trees). +- Within a function, passes run **sequentially** so cascade order is deterministic. +- Cross-function effects (inlining, global constant propagation) use a **module-level fixpoint**: parallel per-function passes → deterministic module phase → re-enqueue only affected functions. +- Epoch counters prevent stale trait reads in parallel workers. + +### 4.4 Comptime evaluator + +An interpreter over HIR subgraphs. Eligible only if: no side effects, primitives only, no global mutation, no I/O, and within loop/recursion budgets. Otherwise it bails and defers to VIR. Outputs `const_value`, then `is_comptime` + `is_constant`. + +The frontend `static` attribute **seeds** the evaluator. HIR trusts `static` as "this is compile-time" but still validates it. + +**Budgets (configurable; defaults):** + +| Budget | Default | +|---|---| +| `comptime.max_iterations` | 4096 | +| `comptime.max_recursion_depth` | 256 | +| `comptime.max_steps` | 1,000,000 | +| `comptime.max_aggregate_elements` | 65,536 | +| `hir.max_rewrites_per_function` | 100,000 | +| `hir.max_rewrites_per_module` | 1,000,000 | + +All are overridable via the API and CLI. + +### 4.5 Defer + +HIR carries a real high-level `defer` statement. The optimizer inlines the deferred call at every scope-exit path and then erases the `defer` marker — zero runtime tax. + +### 4.6 Optimization-level behavior + +| Level | HIR | VIR | +|---|---|---| +| `-O0` | traits validated only | off | +| `-O1` | builtin folds + discovery | off | +| `-O2` | full cascade + comptime | **on** | +| `-O3` | + special AST transforms (loop-unfold/vectorize → constant-unfold → comptime) | on | +| `-Ofast` | = `-O3` + `-march=native` + fast-math | on | +| `-Oz` | size-tuned | on | + +**Compiler-friendly C is emitted at ALL optimization levels.** `-O3` merely adds more aggressive AST transforms. + +--- + +## 5. Lowering (HIR → VIR) + +**Role:** the "middleman". Pure translation + SSA construction; **no discovery**. + +### 5.1 Contract + +- **Input:** annotated HIR with **complete, epoch-valid** traits. +- **Output:** VIR — basic blocks, phi-nodes, def-use, SSA. +- Lowering **validates trait completeness**. A missing or contradictory trait raises an **internal-compiler-error** diagnostic (see §11). +- Deterministic; parallelizable per function. + +### 5.2 Form + +Lowering is a **streaming recursive interpreter** over the HIR tree ("tcc-for-VIR"): + +- Structured HIR (`if`/`while`/`for`/`switch`/`block`) → blocks + terminators (`br`/`condbr`/`switch`/`ret`/`unreachable`). +- Expressions → temporaries + instructions. + +### 5.3 SSA construction + +SSA is built with the **Braun et al. sealed-block algorithm**: one pass, on-the-fly phi insertion, no separate dominance-frontier pass. A loop header is sealed once its back-edge is emitted. + +### 5.4 Memory model (trait-driven hybrid) + +| Condition | Representation | +|---|---| +| scalar ∧ ¬`is_addressed` ∧ ¬`escapes` ∧ ¬`is_runtime_mutable` | pure SSA value | +| addressed / escaping / runtime-mutable / aggregate | `alloca` + explicit `load`/`store` | +| volatile / atomic | forced memory, never promoted | + +VIR's `mem2reg`/`SROA` MAY promote memory back to SSA when traits confirm safety. + +### 5.5 Phi + +VIR has an explicit LLVM-style `Phi` instruction whose operands are `(value, predecessor-block)` pairs. + +### 5.6 Identity & traits + +A `LoweringMap` (HIR node → VIR entity(ies)) is built during lowering. Each VIR entity carries its own `Trait`, populated from the source HIR node. HIR Relationship endpoints are rewritten to VIR entities via the map. VIR depends only on `vct.traits` + node identity. + +### 5.7 Optimization barriers + +`complex`/`NoOptimize` nodes are lowered inside a **region-level** optimization barrier. VIR passes MUST NOT rewrite across it. + +### 5.8 Defer fallback + +If HIR optimization is off (`-O0`/`-O1`), lowering itself expands `defer` at every scope exit. + +### 5.9 VIR type system + +More minimal than HIR but still fleshed out: integers with width, floats, pointers, aggregates, function types, `void`. + +### 5.10 Output invariants + +Well-formed SSA: + +- every use is dominated by its def, +- every block is terminated, +- phi arity equals predecessor count, +- a single def per value. + +Verified in debug builds (§12). + +--- + +## 6. VIR Optimizer + +**Role:** the heavy-lifting half. Reads traits only, transforms SSA, performs **no discovery**. Runs at `-O2`/`-O3`/`-Ofast`/`-Oz`. + +### 6.1 Contract + +- **Input:** lowered VIR (well-formed SSA/CFG) + complete traits. +- **Output:** optimized VIR, still in SSA. +- **Strict trait consumer:** a transform is enabled by an attribute/relationship or requested by a Request. VIR MUST NOT assume a fact absent from traits. + +### 6.2 Framework + +```d +interface VirPass { + string name(); + void run(VirModule m, PassContext ctx); +} +``` + +**Analyses:** `DominatorTree`, `PostDominatorTree`, `LoopInfo`, `AliasInfo`, `DefUse`, `CallGraph`, `RangeInfo`. Passes declare which analyses they preserve; the manager invalidates the rest (**fine-grained**, not invalidate-all). Epoch counters catch stale trait reads. + +### 6.3 Default pipeline (`-O2`) + +A module pass manager interleaving function passes and IPA: + +1. **Canonicalize:** `mem2reg`, `SROA`, `instcombine`, `simplifyCFG`, `early-CSE`, `DCE`. +2. **Scalar:** `GVN`, `SCCP`, `LICM`, `indvars`, `reassociation`. +3. **IPA:** inliner (driven by `Inline`/`AlwaysInline`/`NoInline`/`ColdCall`/`LikelyCall`), global `DCE`, `IPSCCP`, function-attribute propagation. +4. **Loops:** `unroll`/`interleave`/`peel`/`rotate`/`unswitch`/`distribute`/`fuse`/`jam`, each gated by its Request. +5. **Vectorize:** loop + SLP, gated by `Vectorize`/`LoopInvariant`/`Range`/`NoAlias`/`Restrict`. +6. **Memory:** alias-driven `DSE`, GEP simplification, load widening. +7. **Control flow:** block layout (`HotPath`/`ColdPath`), tail-call formation (`TailCall`/`MustTail`/`NoTail`), unreachable pruning, jump threading. +8. **Codegen prep:** canonicalize for the backend; remains SSA. + +`-O3`/`-Ofast` raise aggression. `-Oz` is size-first (most unrolling/vectorization off). + +### 6.4 Requests + +Each Request yields a `RequestResult` (§3.2). Denials carry a `DenyReason` + one-line note. + +### 6.5 Alias analysis + +Synthesized from `NoAlias`/`Restrict`/`NoCapture`/`Dereferenceable`/`ReadOnly`/`WriteOnly` + provenance — no guessing. + +### 6.6 Barriers + +Region optimization barriers (`complex`/`NoOptimize`, §5.7) are opaque; passes skip across them. + +### 6.7 Budgets + +`vir.max_iterations`, `vir.max_pipeline_rounds` (configurable). + +### 6.8 Resolutions + +- A **Strong-request denial is a hard error** when the backend is capable and the reason is `Illegality`/`ContradictsTrait`; it is a **warning** for `Unsupported`/`NoTargetSupport`. +- The pipeline is a **fixed canonical pipeline** (LLVM-style), not adaptive request-driven ordering. +- **Inlining is a VIR/IPA transform** (HIR only requests it). +- `RangeInfo` is **lightweight**, seeded by `Range`/`Assume` traits — no full ScalarEvolution. +- Analysis invalidation is **fine-grained** (preserved-analyses sets). + +--- + +## 7. C Backend (`vct.backend.c`) + +**Role:** VIR → C17. + +### 7.1 Contract + +- **Input:** optimized VIR (still SSA, traits attached). +- **Output:** one C17 translation unit **per function**. +- Emission is per-function independent → parallel; the driver concatenates deterministically. +- Two-pass emission: forward declarations, then definitions. + +### 7.2 Out-of-SSA + +Boissinot et al., *"Revisiting Out-of-SSA Translation"* — handles the lost-copy and swap problems. **Critical edges are split before** out-of-SSA. + +### 7.3 Compiler-friendly C + +- **Structured control-flow reconstruction** (`if`/`else`/`while`/`for`/`do`/`switch`) for reducible CFGs; `goto` fallback for irreducible. +- **Qualifiers from traits:** `restrict` (`Restrict`/`NoAlias`), `const` (`ReadOnly`/`NoWrite`), `_Noreturn` (`NoReturn`), `cold`/`hot` (`ColdPath`/`HotPath`), `static inline` / `noinline`. +- **Hints:** `__builtin_expect`; `__builtin_assume_aligned` / `_Alignas` (`Align`); `__builtin_unreachable` (`Unreachable`); `__builtin_assume` (clang, `Assume`); `#pragma GCC ivdep` / `#pragma clang loop vectorize(enable)` (`Vectorize`); `pure`/`const` (`NoRead`/`NoWrite`). +- **`NoOptimize`/`complex` barrier:** a compiler fence — `__asm__ __volatile__("" ::: "memory")` plus an opaque `noinline` call. + +### 7.4 Types & layout + +`intN_t`/`uintN_t`, `float`/`double`, `T*`, `struct`/`union`/arrays, function pointers, `void`. VIR aggregate GEPs use **field indices**, not byte offsets; the C compiler picks layout. An explicit **layout trait** pins the ABI (emits padding + `_Static_assert`). + +### 7.5 Naming + +Deterministic mangling. Exported names preserved; internal names sanitized and uniquified. A reserved-word/collision prefix table. + +### 7.6 Intrinsics + +Mostly `__builtin_*` (GCC/Clang), plus a small set of portable fallback helpers for what builtins do not cover. + +### 7.7 Debug mapping + +`#line` directives back to frontend source, flag-gated (on for debug builds, off for release). The frontend registers its source map with `vct.diag`. + +### 7.8 ABI + +Exported functions match the C ABI; internal functions are `static`; struct-by-value follows the C compiler's ABI. `MustTail` requires `[[clang::musttail]]` on Clang; on GCC a `MustTail` request is a **hard error**. + +--- + +## 8. LLVM Backend (`vct.backend.llvm`) + +**Role:** VIR → textual LLVM IR, **post** VIR optimization. Pure translation; no optimization or discovery. Output is a self-contained `.ll` module. libVCT never invokes `llvm-as`/`opt`/`llc` or links LLVM — that is the frontend's job. Deterministic per-function emission with stable concatenation. + +### 8.1 SSA mapping + +VIR is already SSA → **1:1 mapping**: instruction → instruction, block → block, `Phi` → `phi`, terminators, `alloca`/`load`/`store`. **No out-of-SSA** (the opposite of the C backend). + +### 8.2 Types + +`iN` integers, `float`/`double`, **opaque `ptr`** (not typed), `struct`/array (VIR field indices → LLVM struct indices), `void`, function types. + +### 8.3 Traits → LLVM attributes/metadata + +| Trait | LLVM | +|---|---| +| `Restrict`/`NoAlias` | `noalias` + `!alias.scope`/`!noalias` | +| `ReadOnly`/`WriteOnly`/`NoRead`/`NoWrite` | `readonly`/`writeonly` | +| `NonNull` | `nonnull` | +| `Align(n)` | `align n` | +| `Range` | `!range` on loads | +| `Assume` | `llvm.assume` | +| `NoInline`/`AlwaysInline` | `noinline`/`alwaysinline` | +| `TailCall`/`MustTail`/`NoTail` | `tail`/`musttail`/`none` | +| `NoReturn` | `noreturn` | +| `HotPath`/`ColdPath` | `!prof` | +| `Vectorize` | `!llvm.loop.vectorize.enable` | +| `Unroll(n)` | `!llvm.loop.unroll.count` | +| `Unreachable` | `unreachable` | +| `NoUndef` | reserved; only meaningful here | + +### 8.4 Intrinsics + +`memcpy`/`memset`/`memmove`, `llvm.sqrt.*`, `llvm.fabs.*`, `llvm.ctpop.*`, `llvm.fshl.*`, saturating/overflow ops. + +### 8.5 Module scaffolding + +Target triple (from `-target`) + derived target datalayout, so the module is self-contained. Declarations precede definitions. + +### 8.6 Resolutions + +- Pin **LLVM 18+** only (opaque pointers, current metadata). +- **No `!llvm.dbg`** in v1; debug only on the C path via `#line`. +- Emit **text `.ll` only**; the caller runs `llvm-as`. +- Emit **real** `llvm.assume` / `!range`. + +--- + +## 9. Driver & CLI + +### 9.1 Role split + +- **`vct.driver`** — library-level orchestrator: takes emitted C TUs, invokes the C compiler, collects `.o`. Owns the `-llvm` path (returns LLVM IR, never touches `cc`) and the `-mangled` path. +- **`vct.cli`** — thin wrapper: parses flags into `Config`, drives the whole pipeline (read IR → HIR opt → lower → VIR opt → `backend.c` → driver), formats diagnostics, sets exit codes. GC-allocated; holds no optimizer state. + +### 9.2 Pipeline + +``` +read input + -> (HIR opt if enabled) + -> lower + -> (VIR opt if enabled) + -> backend.c -> driver -> .o +``` + +`-S` stops after C; `-emit-llvm`/`-llvm` stop after LLVM IR. + +### 9.3 Flag surface (v1) + +| Group | Flags | +|---|---| +| **Optimize** | `-O0` `-O1` `-O2` `-O3` `-Ofast` `-Oz` | +| **Target** | `-march=` (→ cflags), `-mcpu=`, `-target ` | +| **Modes** | `-S` (C only), `-emit-llvm`/`-llvm`, `-mangled` (cosmopolitan) | +| **Toolchain** | `-cc=` (default auto), `-cflags="..."`, `-j`, `-save-temps` | +| **LTO** | `-flto[=full\|thin]`, `-ffat-lto-objects` | +| **Debug** | `-g`, `--dump-hir`, `--dump-vir`, `--verify`, `--time-passes`, `--stats` | +| **Output** | `-o ` | +| **Warnings** | `-Wall -Werror -Wextra` **always on**; `-Wno-error` escape hatch; `-w` to silence | +| **Diagnostics** | `--remarks`, `--diagnostics=json` | +| **Pass toggles** | every pass exposed as `-f` / `-fno-` | + +### 9.4 cc invocation + +``` +cc -std=c17 -c -o -Wall -Werror -Wextra +``` + +One process per TU, bounded by `-j`. The compiler is discovered via `$CC` then `PATH`; its version/dialect is probed once and drives the backend's pragma/builtin choices (§7.3, §7.6). + +### 9.5 Error handling + +`cc` stderr is captured. `#line` directives (§7.7) map C errors back to VIR/source; they are re-emitted via `vct.diag`. A `cc` failure produces a diagnostic and a nonzero exit. + +### 9.6 Determinism + +Stable TU ordering and flag ordering. Temp files live under a GC-managed temp dir (cleaned unless `-save-temps`). + +### 9.7 `-mangled` + +Switches the toolchain to `cosmocc`/cosmopolitan libc and produces an Actually Portable Executable. This mode **does link** — the opt-in exception to "caller links". + +### 9.8 Resolutions + +- Driver compiles to `.o` only; a single `-o` merges per-function objects with an `ld -r` relocatable link; a recommended link line / response file is emitted for the caller. `-mangled` is the sole mode producing a final executable. +- The CLI also reads a **textual HIR/VIR format** for testing/tools/round-trips; frontends still embed the library. +- **Every pass** is exposed as a `-f`/`-fno` toggle. +- `-Wall -Werror -Wextra` default; `-Wno-error` escape hatch; `-Werror` applies to libVCT-generated C. + +--- + +## 10. Diagnostics & Source Maps (`vct.diag`) + +### 10.1 Central service + +One central diagnostics service; every module emits through it, nothing prints directly. GC-allocated. + +```d +struct Diagnostic { + Severity severity; // Error | Warning | Note | Remark | Ice + DiagCode code; // stable namespaced, e.g. VCT1002 + string message; + Span primary; + Span[] notes; + Suggestion[] suggestions; +} +``` + +### 10.2 Source maps + +The frontend registers `SourceLocation{file,line,col,len}` against HIR nodes; VIR resolves via the `LoweringMap` (§5.6). Degradation to no-location is graceful. The C backend emits `#line` (§7.7); `cc` errors are parsed and re-mapped (§9.5). + +### 10.3 Channels + +- `DiagnosticConsumer` callback (canonical for embedded use). +- Human-readable CLI (color when tty). +- JSON (`--diagnostics=json`, **versioned schema**, shipped in v1). + +### 10.4 Optimizer remarks + +`--remarks` (off by default; automatically enabled at `-O3`) emits per-decision remarks tied to the trait pipeline, e.g.: + +- `HIR: replaced 3 uses of node#412 (comptime 64)` +- `VIR: denied Inline on @foo — CostModel (callee 2.4x size budget)` + +### 10.5 ICE policy + +Internal compiler errors include: missing/contradictory trait at lowering (§5.1), VIR invariant violated in debug (§5.10), and a Strong request denied for `Illegality`/`ContradictsTrait` (§6.8). + +- **Release:** return a failure result + ICE diagnostic with a module-dump request. +- **Debug:** assert. + +Never silently continue. + +### 10.6 Exit codes + +| Code | Meaning | +|---|---| +| 0 | success | +| 1 | diagnostics present | +| 2 | usage/config error | +| 3 | ICE | + +### 10.7 Resolutions + +- The trait-mismatch warning **is** promoted to error under `-Werror` (frontends can opt out with `-Wno-error=`). +- ICE in release = diagnostic + error result. +- Remarks: both opt-in `--remarks` **and** automatically enabled at `-O3`. +- Ship a versioned JSON diagnostics format in v1. + +--- + +## 11. Testing & Verification + +### 11.1 Core constraint + +VIR is a strict trait consumer and cannot be tested without a valid HIR fixture. The test strategy is anchored on a fixture layer. + +### 11.2 Layers + +1. **Unit tests** — D `unittest` per module. +2. **HIR test-builder harness** (`vct.test.hirbuild`) — a fluent builder constructing HIR modules + traits without a frontend. This is the fixture layer for HIR-opt, lowering, VIR-opt, and both backends. +3. **Textual round-trip** — property: `print(parse(print(m))) == print(m)`. +4. **FileCheck-style tests** — `CHECK` / `CHECK-NOT` / `CHECK-NEXT` directives. +5. **Verifier** — structural SSA (§5.10) + trait consistency, via `--verify`; asserted after every pass in debug. +6. **Differential** — the VIR reference interpreter vs the compiled `.o`; the C path vs the LLVM path. +7. **Fuzz** — random HIR/trait combinations; property: terminates, verifier clean, output compiles. +8. **Matrix** — gcc + clang × `-O0`..`-Ofast`/`-Oz` × C/LLVM. +9. **Perf benchmarks** — codegen quality is the headline; compile time tracked via `--time-passes`/`--stats`. + +### 11.3 VIR reference interpreter + +Approved as **v1 scope**. It is the differential oracle: the same VIR is run through the interpreter and through the compiled `.o`, and results are compared. + +### 11.4 Determinism + +Every pipeline run is executed twice in CI; output must be byte-identical. + +### 11.5 Regression corpus + +A corpus of input IR + expected FileCheck patterns; CI runs the full matrix. + +### 11.6 Resolutions + +- FileCheck-style matcher (not exact golden files). +- Build the VIR reference interpreter in v1 as a differential oracle. +- Verifier always-on in debug builds; `--verify` opt-in in release. +- Random-HIR + random-trait fuzzers in v1. + +--- + +## 12. Build, Packaging & Public API Lifecycle + +### 12.1 Language & toolchain + +Written in D. Primary compiler: **LDC2**; **DMD** supported as a secondary build target. **Zero runtime dependencies** — only a D compiler to build, and a system C compiler discovered at runtime for the C path. + +### 12.2 Build system + +`dub` for normal development, **plus** a plain `build.d`/Makefile path for no-dub builds. Both MUST produce identical artifacts. + +### 12.3 Artifacts + +- `libvct.a` (static) and `libvct.so` (shared). +- `vct` CLI binary. +- `vctc.h` — C API header. +- `vct.test.hirbuild` — shipped publicly. + +### 12.4 API surfaces + +- **Native D API** (primary, full feature set). +- **Thin `extern(C)` C API** (v1): opaque handles for `Context`/`Module`/`Builder`/`Config`, functions to build IR, run the pipeline, and query diagnostics. No logic lives in the shim. + +### 12.5 Module visibility + +- **Public:** `vct.ir.hir`, `vct.traits`, `vct.context`, `vct.backend.c` entry, `vct.backend.llvm` entry, `vct.driver`, `vct.diag`, `vct.test.hirbuild`. +- **Internal:** `vct.ir.vir`, `vct.lower`, `vct.hir.opt`, `vct.vir.opt`, `vct.comptime` internals. + +### 12.6 Memory ownership & lifecycle + +- A `Context` owns one or more **arenas**; an arena is the unit of reclamation. Default: one arena per `Module`, freed wholesale. +- All IR and optimizer objects are arena-owned and non-GC; passes mutate in place under epoch guards. +- GC is permitted only on cold paths (driver, diagnostics, CLI). +- `Context` accepts an **`ArenaAllocator` interface** so embedders can supply backing memory. + +### 12.7 `Config` + +One struct holds opt level, target (`-march`/`-mcpu`/`-target`), all budgets, pass toggles, warning policy, cc selection, and mode (`-S`/`-llvm`/`-mangled`). The CLI is a pure function from argv to `Config`; the library takes a `Config`. Programmatic callers bypass the CLI. + +### 12.8 Threading contract + +Thread-safe when each compilation unit has its own `Context`/arenas; no shared mutable global state. Intra-module parallelism is internal and bounded by `-j`. + +### 12.9 Versioning + +- Library semver. +- **Textual IR format version** — separate and independently versioned. +- **Trait vocabulary version** — adding attributes/requests is backward-compatible; changing semantics is a major bump. Experimental passes are gated behind feature flags. + +### 12.10 Determinism guarantee + +Same `Config` + same input ⇒ byte-identical output, independent of thread count. + +### 12.11 Resolutions + +- C API in v1 (thin shim). +- Custom `ArenaAllocator` hooks exposed. +- LDC2 primary + DMD secondary. +- `dub` + no-dub build path. + +--- + +## 13. Worked End-to-End Example + +This example is **normative** for trait/request semantics. + +Input program: + +``` +fn foo(x: int, y: int) -> int { + return x + y; +} + +fn main() -> int { + x = 4; + y = x + 4; + z = foo(x, y); + a = sqrt(z); + println(a); +} +``` + +The HIR cascade proceeds as follows: + +1. HIR comptime-evaluates `x = 4` → `is_comptime`, `is_constant`, `const_value = 4`. +2. `y = x + 4` unfolds to `y = 8` → `is_comptime`. +3. `foo(x, y)` is proven small → HIR issues an `Inline` **request**; `z` becomes `8 + ...` constant → comptime. +4. `a = sqrt(z)` → `a = sqrt(8)` comptime → `a = 3.4641016151377544`. +5. The whole program collapses to `println(3.4641016151377544)`. + +VIR then optimizes the `println` call. The emitted C is effectively a single call with the folded constant. This demonstrates trait propagation HIR → VIR and the request lifecycle (`Inline` accepted). + +--- + +## 14. Open Questions & Future Work + +- **Debug info on the LLVM path** (`!llvm.dbg`) — deferred past v1. +- **Windows / MSVC support** — out of scope for v1. +- **LLVM bitcode emission** — deferred; caller runs `llvm-as`. +- **Incremental / cached compilation** — content-hash `.o` caching is out of scope for v1. +- **Cross-compilation targets** beyond `-target` passthrough and cosmopolitan — future work.