diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..5301281 --- /dev/null +++ b/.gitignore @@ -0,0 +1,2 @@ +# Local working artifacts (specs, plans) — not version-controlled +docs/superpowers/ diff --git a/docs/superpowers/specs/2026-09-13-libvct-design.md b/docs/superpowers/specs/2026-09-13-libvct-design.md deleted file mode 100644 index 7d62cdd..0000000 --- a/docs/superpowers/specs/2026-09-13-libvct-design.md +++ /dev/null @@ -1,788 +0,0 @@ -# libVCT — Vox C Transpiler Library: Architecture Specification - -- **Status:** Draft (pending final review) -- **Date:** 2026-09-13 -- **Scope:** Whole-library architecture — component boundaries and interfaces. Subsystem internals are deferred to later specs. -- **Normative language:** `MUST`, `SHOULD`, `MAY` per RFC 2119. - ---- - -## 1. Overview - -libVCT is an LLVM-inspired compiler library written in D. It consumes an IR, optimizes it, and transpiles it into C17, then drives a system C compiler to produce an object file. An alternate backend emits textual LLVM IR instead of C. - -The library is organized around a **two-stage IR**: - -- **HIR** — a high-level, tree-shaped AST that is the frontend's output and is **annotated in place** with *traits*. HIR does discovery and small, cheap folds. -- **VIR** — a low-level SSA/CFG instruction IR. VIR does the heavy lifting and performs **no discovery**; it is a strict consumer of HIR-provided traits. - -The design thesis: because HIR hands VIR a fully annotated "dictionary" of facts, VIR can skip analysis and go straight to transformation. HIR's cheap, cascading folds are parallelizable; the resulting VIR is simpler and faster to optimize. The net goal is LLVM-class codegen quality with materially lower compile time. - -### 1.1 Goals - -1. **Codegen quality is the headline.** Generated code quality takes priority over compile speed. -2. **Two-stage optimization.** HIR performs small folds and trait derivation; VIR performs large transformations. -3. **Trait-driven VIR.** VIR never re-analyzes; every fact it uses is an attribute, relationship, or request supplied by HIR. -4. **Parallel, cascading HIR.** HIR passes run per-function in parallel and cascade to a fixpoint. -5. **Compiler-friendly C.** The C backend always emits C written to be pattern-matched by GCC/Clang. -6. **Deterministic output.** Same input + same config ⇒ byte-identical output, regardless of thread count. - -### 1.2 Non-goals (v1) - -- **Linking.** libVCT produces object files; linking is the caller's job. The sole exception is `-mangled` (cosmopolitan) mode. -- **Windows / MSVC.** Linux/POSIX-first; gcc and clang only. -- **LLVM bitcode.** The LLVM backend emits textual `.ll` only; the caller runs `llvm-as`. -- **LLVM debug metadata.** No `!llvm.dbg` in v1; debug mapping exists on the C path via `#line`. -- **A hand-authored VIR.** VIR is internal and intentionally hostile to hand-authoring. A textual form exists solely for tools and round-trip tests (§9.8); it is not a supported authoring surface. -- **Consuming LLVM IR or Vox's existing IR.** libVCT owns its IR definition. - -### 1.3 Glossary - -| Term | Meaning | -|---|---| -| **HIR** | High-level IR: the frontend AST, annotated in place with traits. | -| **VIR** | Vox IR: a low-level SSA/CFG instruction IR. | -| **Trait** | The union of Attribute, Request, and Relationship attached to a node. | -| **Attribute** | A fact about a node (type, value, mutability, ...). | -| **Request** | An attribute-derived suggestion from HIR to VIR. | -| **Relationship** | A directed edge between nodes, traceable up/down. | -| **Cascade** | The chain of HIR rewrites triggered by one fold. | -| **Lowering** | The "middleman" that translates annotated HIR into VIR SSA. | -| **Remark** | An optimizer decision record emitted for diagnostics. | - ---- - -## 2. Component Architecture - -### 2.1 Modules - -| Module | Responsibility | Allocation | Visibility | -|---|---|---|---| -| `vct.ir.hir` | HIR AST node types, Trait model, builder/emitter API | arena | **public** | -| `vct.ir.vir` | VIR SSA/CFG types, VIR type system, instruction set | arena | internal | -| `vct.traits` | Attribute / Request / Relationship definitions + queries | arena | **public** | -| `vct.comptime` | Comptime evaluator / constant-folding engine | arena | internal | -| `vct.hir.opt` | HIR pass framework + passes, parallel cascade | arena | internal | -| `vct.lower` | HIR → VIR lowering + SSA construction | arena | internal | -| `vct.vir.opt` | VIR pass framework + heavy passes | arena | internal | -| `vct.backend.c` | VIR → C17 emitter (out-of-SSA, compiler-friendly C) | arena + GC | **public entry** | -| `vct.backend.llvm` | VIR → textual LLVM IR emitter | arena + GC | **public entry** | -| `vct.driver` | cc invocation, flag assembly, `.o` emission, LTO, cosmopolitan | GC | **public** | -| `vct.cli` | Thin command-line wrapper | GC | **public** | -| `vct.diag` | Diagnostics, source maps, error reporting | GC | **public** | -| `vct.context` | Context / arena ownership, `Config` | GC + arena | **public** | -| `vct.test.hirbuild` | Fluent HIR test-builder harness | arena | **public** | - -### 2.2 Data flow - -``` -frontend --build--> HIR - --hir.opt (parallel, cascading)--> annotated HIR - --lower--> VIR (SSA/CFG) - --vir.opt--> optimized VIR - --backend.c--> C17 source --driver--> .o (default path) - -optimized VIR --backend.llvm--> textual LLVM IR (caller links LLVM) -``` - -### 2.3 Public interfaces - -1. **HIR builder (frontend-facing).** Construct nodes, set types/values, attach frontend-supplied traits (`static`, hints), and finish a module. -2. **Trait query (VIR-facing).** Read attributes and walk relationships. This is the **only** channel through which VIR obtains facts. -3. **Backend (driver-facing).** Consume optimized VIR and emit C (or textual LLVM IR). - -### 2.4 Load-bearing boundaries - -- **`vct.traits` is the contract.** Both HIR and VIR depend on it. VIR depends on nothing else from HIR except **node identity**. -- **`vct.lower` is the SSA construction site.** Phi-nodes, dominance, and the memory model all land here; it is the most algorithmically dense module. - ---- - -## 3. The Trait Model - -A `Trait` is the union of three sub-structures hung on HIR nodes, and the entire vocabulary VIR may reason with: - -```d -struct Trait { - Attribute attr; - Request[] reqs; - Relation[] rels; -} -``` - -Traits are **exposed publicly**. A frontend MAY **suggest** traits to HIR; HIR validates suggestions and fills in correct values. Derived traits are authoritative and VIR trusts them blindly. - -### 3.1 Attributes - -Attributes are facts about a node, with **two origins in one namespace**: - -- **Suggested** — set by the frontend at build time. HIR **validates**; on mismatch it emits a warning (promoted to error under `-Werror`) and **overwrites** with the correct value. -- **Derived** — produced by the HIR optimizer during the cascade. Authoritative. - -Every attribute carries an `attr_source` bit recording whether it was suggested or derived. - -**v1 attribute set:** - -| Attribute | Meaning | -|---|---| -| `ty` | The node's VIR type. | -| `const_value` | Present iff the value is known at compile time. | -| `is_static` | Frontend-declared compile-time value. | -| `is_comptime` | HIR-proven compile-time value. | -| `is_constant` | Value is constant (not necessarily comptime). | -| `is_used` | Node has at least one use. | -| `is_mutably_used` | Node is mutated through at least one use. | -| `is_addressed` | Address taken by pointer/reference. | -| `escapes` | Value escapes its defining scope. | -| `is_runtime_mutable` | May change at runtime. | -| `may_change_at_runtime` | The observable value may differ between reads/executions (external state). Distinct from `is_runtime_mutable`, which means the storage is written at runtime. | -| `complex` | Frontend-provided; gates the `NoOptimize` request. | - -### 3.2 Requests - -Requests are **attribute-derived** suggestions from HIR to VIR — never ad-hoc. Each has a **strength**: - -- `Soft` — a suggestion VIR MAY decline. -- `Strong` — violating it is likely a bug (e.g., `NoInline`, `MustTail`, `AlwaysInline`, `NoOptimize`). - -Every request produces a `RequestResult`: - -```d -struct RequestResult { - Request req; - bool accepted; - DenyReason reason; - string note; // one-line human-readable explanation -} -``` - -A denial MUST carry a `DenyReason` enum (for tooling) and a one-line `note`. - -```d -enum DenyReason { - CostModel, // profitable only under a different cost model - Illegality, // transform would be incorrect - AlreadyDone, // no-op: the goal already holds - Unsupported, // VIR does not implement it yet - ContradictsTrait, // conflicts with a stronger fact - TooLarge, // exceeds a size budget - NoTargetSupport, // backend/target cannot express it -} -``` - -#### 3.2.1 Request catalog (v1) - -**Inlining / call edges:** `Inline`; `AlwaysInline` (Strong); `NoInline` (Strong); `TailCall`; `MustTail` (Strong); `NoTail` (Strong); `Devirtualize`; `ColdCall`; `LikelyCall`; `UnlikelyCall`. - -**Loops:** `Vectorize`; `Unroll(factor)`; `Interleave(count)`; `Peel(count)`; `Distribute`; `Fuse`; `Jam`; `Unswitch`; `Rotate`; `LICM` (action); `LoopInvariant` (property); `MustProgress`. - -**Memory effects:** `NoAlias`; `Restrict`; `NonNull`; `Align(n)`; `ReadOnly`; `WriteOnly`; `NoRead`; `NoWrite`; `NoCapture`; `Dereferenceable(n)`; `Constant`; `NoUndef` (reserved); `Prefetch`. - -**Control flow:** `HotPath`; `ColdPath`; `Unreachable`; `NoReturn`. - -**Assumptions:** `Assume(pred)`; `Range(lo, hi)`. - -**Optimization control:** `NoOptimize` (Strong; requested **only** when the frontend sets `complex` on the node). - -#### 3.2.2 Request conflict rules - -- `Strong` beats `Soft`. -- Two conflicting `Strong` requests (e.g., `Inline` vs `NoInline`, `TailCall` vs `NoTail`) ⇒ HIR emits a diagnostic and **rejects** the frontend suggestion. -- `LoopInvariant` (a proven property VIR may exploit) and `LICM` (an explicit request to hoist) are **distinct** and both retained. -- `NoRead`/`NoWrite` are **primitives on an access edge**; `ReadOnly`/`WriteOnly` are **region-level facts** derived from them. All four are retained, with the scope difference documented. - -### 3.3 Relationships - -Relationships are directed edges HIR leaves for the non-constant world. They are traceable **up and down** to a terminal node. - -**Flags:** - -| Flag | Meaning | -|---|---| -| `is_offspring` | Cannot trace up. | -| `is_ancestor` | Cannot trace down. | -| `is_common` | Branch point; VIR must choose a direction. | -| `has_siblings` | Connects down to 2+ nodes. | -| `is_apex` | Part of a cycle (`a → b → c → d → a`), usually from polymorphism. | - -**Relation attributes (small set):** `was_changed`; `is_mutable`; `is_pointer`. - -An apex SHOULD normally be ironed out by HIR; residual apexes typically arise from polymorphism. - -### 3.4 Representation & staleness - -- **Representation (resolved):** an **embedded dense `Trait` struct** on every node is primary, plus an **optional sparse side-channel** for rare / frontend-extensible attributes. -- **Staleness (resolved):** **epoch/generation counters** with dirty propagation along relationships. A trait is valid only for the node's current epoch. This survives parallel HIR passes. - ---- - -## 4. HIR Optimizer - -**Role:** discovery + small folds + trait derivation. HIR hands VIR a "dictionary". - -### 4.1 Pass framework - -```d -interface HirPass { - string name(); - void run(HirFunction fn, HirContext ctx); -} -``` - -Passes mutate the annotated AST in place. Successful rewrites emit **triggers**: - -```d -enum Trigger { - ConstantUnfolded, - UsesReplaced, - NodeDeleted, - TraitChanged, - StaticDiscovered, -} -``` - -A **worklist scheduler** collects `(pass, node)` pairs, dedupes, and runs to a fixpoint. A per-function step budget guarantees termination. - -### 4.2 The cascade - -Canonical chain: - -``` -4 * 16 - -> const 64 - -> mark is_comptime / is_constant / const_value = 64 - -> prove not addressed / not runtime-mutable - -> replace all uses with literal 64 - -> node dead -> delete - -> downstream nodes now constant -> enqueue -``` - -Each hop emits a trigger; the worklist drains until nothing new is provable. **HIR never inlines** — it only proves smallness and issues an `Inline` request to VIR. - -### 4.3 Parallelism & determinism - -- Parallelize **per function** (independent HIR trees). -- Within a function, passes run **sequentially** so cascade order is deterministic. -- Cross-function effects (inlining, global constant propagation) use a **module-level fixpoint**: parallel per-function passes → deterministic module phase → re-enqueue only affected functions. -- Epoch counters prevent stale trait reads in parallel workers. - -### 4.4 Comptime evaluator - -An interpreter over HIR subgraphs. Eligible only if: no side effects, primitives only, no global mutation, no I/O, and within loop/recursion budgets. Otherwise it bails and defers to VIR. Outputs `const_value`, then `is_comptime` + `is_constant`. - -The frontend `static` attribute **seeds** the evaluator. HIR trusts `static` as "this is compile-time" but still validates it. - -**Budgets (configurable; defaults):** - -| Budget | Default | -|---|---| -| `comptime.max_iterations` | 4096 | -| `comptime.max_recursion_depth` | 256 | -| `comptime.max_steps` | 1,000,000 | -| `comptime.max_aggregate_elements` | 65,536 | -| `hir.max_rewrites_per_function` | 100,000 | -| `hir.max_rewrites_per_module` | 1,000,000 | - -All are overridable via the API and CLI. - -### 4.5 Defer - -HIR carries a real high-level `defer` statement. The optimizer inlines the deferred call at every scope-exit path and then erases the `defer` marker — zero runtime tax. - -### 4.6 Optimization-level behavior - -| Level | HIR | VIR | -|---|---|---| -| `-O0` | traits validated only | off | -| `-O1` | builtin folds + discovery | off | -| `-O2` | full cascade + comptime | **on** | -| `-O3` | + special AST transforms (loop-unfold/vectorize → constant-unfold → comptime) | on | -| `-Ofast` | = `-O3` + `-march=native` + fast-math | on | -| `-Oz` | size-tuned | on | - -**Compiler-friendly C is emitted at ALL optimization levels.** `-O3` merely adds more aggressive AST transforms. - ---- - -## 5. Lowering (HIR → VIR) - -**Role:** the "middleman". Pure translation + SSA construction; **no discovery**. - -### 5.1 Contract - -- **Input:** annotated HIR with **complete, epoch-valid** traits. -- **Output:** VIR — basic blocks, phi-nodes, def-use, SSA. -- Lowering **validates trait completeness**. A missing or contradictory trait raises an **internal-compiler-error** diagnostic (see §10.5). -- Deterministic; parallelizable per function. - -### 5.2 Form - -Lowering is a **streaming recursive interpreter** over the HIR tree ("tcc-for-VIR"): - -- Structured HIR (`if`/`while`/`for`/`switch`/`block`) → blocks + terminators (`br`/`condbr`/`switch`/`ret`/`unreachable`). -- Expressions → temporaries + instructions. - -### 5.3 SSA construction - -SSA is built with the **Braun et al. sealed-block algorithm**: one pass, on-the-fly phi insertion, no separate dominance-frontier pass. A loop header is sealed once its back-edge is emitted. - -### 5.4 Memory model (trait-driven hybrid) - -| Condition | Representation | -|---|---| -| scalar ∧ ¬`is_addressed` ∧ ¬`escapes` ∧ ¬`is_runtime_mutable` | pure SSA value | -| addressed / escaping / runtime-mutable / aggregate | `alloca` + explicit `load`/`store` | -| volatile / atomic | forced memory, never promoted | - -VIR's `mem2reg`/`SROA` MAY promote memory back to SSA when traits confirm safety. - -### 5.5 Phi - -VIR has an explicit LLVM-style `Phi` instruction whose operands are `(value, predecessor-block)` pairs. - -### 5.6 Identity & traits - -A `LoweringMap` (HIR node → VIR entity(ies)) is built during lowering. Each VIR entity carries its own `Trait`, populated from the source HIR node. HIR Relationship endpoints are rewritten to VIR entities via the map. VIR depends only on `vct.traits` + node identity. - -### 5.7 Optimization barriers - -`complex`/`NoOptimize` nodes are lowered inside a **region-level** optimization barrier. VIR passes MUST NOT rewrite across it. - -### 5.8 Defer fallback - -If HIR optimization is off (`-O0`/`-O1`), lowering itself expands `defer` at every scope exit. - -### 5.9 VIR type system - -More minimal than HIR but still fleshed out: integers with width, floats, pointers, aggregates, function types, `void`. - -### 5.10 Output invariants - -Well-formed SSA: - -- every use is dominated by its def, -- every block is terminated, -- phi arity equals predecessor count, -- a single def per value. - -Verified in debug builds (§11). - ---- - -## 6. VIR Optimizer - -**Role:** the heavy-lifting half. Reads traits only, transforms SSA, performs **no discovery**. Runs at `-O2`/`-O3`/`-Ofast`/`-Oz`. - -### 6.1 Contract - -- **Input:** lowered VIR (well-formed SSA/CFG) + complete traits. -- **Output:** optimized VIR, still in SSA. -- **Strict trait consumer:** a transform is enabled by an attribute/relationship or requested by a Request. VIR MUST NOT assume a fact absent from traits. - -### 6.2 Framework - -```d -interface VirPass { - string name(); - void run(VirModule m, PassContext ctx); -} -``` - -**Analyses:** `DominatorTree`, `PostDominatorTree`, `LoopInfo`, `AliasInfo`, `DefUse`, `CallGraph`, `RangeInfo`. Passes declare which analyses they preserve; the manager invalidates the rest (**fine-grained**, not invalidate-all). Epoch counters catch stale trait reads. - -### 6.3 Default pipeline (`-O2`) - -A module pass manager interleaving function passes and IPA: - -1. **Canonicalize:** `mem2reg`, `SROA`, `instcombine`, `simplifyCFG`, `early-CSE`, `DCE`. -2. **Scalar:** `GVN`, `SCCP`, `LICM`, `indvars`, `reassociation`. -3. **IPA:** inliner (driven by `Inline`/`AlwaysInline`/`NoInline`/`ColdCall`/`LikelyCall`), global `DCE`, `IPSCCP`, function-attribute propagation. -4. **Loops:** `unroll`/`interleave`/`peel`/`rotate`/`unswitch`/`distribute`/`fuse`/`jam`, each gated by its Request. -5. **Vectorize:** loop + SLP, gated by `Vectorize`/`LoopInvariant`/`Range`/`NoAlias`/`Restrict`. -6. **Memory:** alias-driven `DSE`, GEP simplification, load widening. -7. **Control flow:** block layout (`HotPath`/`ColdPath`), tail-call formation (`TailCall`/`MustTail`/`NoTail`), unreachable pruning, jump threading. -8. **Codegen prep:** canonicalize for the backend; remains SSA. - -`-O3`/`-Ofast` raise aggression. `-Oz` is size-first (most unrolling/vectorization off). - -### 6.4 Requests - -Each Request yields a `RequestResult` (§3.2). Denials carry a `DenyReason` + one-line note. - -### 6.5 Alias analysis - -Synthesized from `NoAlias`/`Restrict`/`NoCapture`/`Dereferenceable`/`ReadOnly`/`WriteOnly` + provenance — no guessing. - -### 6.6 Barriers - -Region optimization barriers (`complex`/`NoOptimize`, §5.7) are opaque; passes skip across them. - -### 6.7 Budgets - -`vir.max_iterations`, `vir.max_pipeline_rounds` (configurable). - -### 6.8 Resolutions - -- A **Strong-request denial is a hard error** when the backend is capable and the reason is `Illegality`/`ContradictsTrait`; it is a **warning** for `Unsupported`/`NoTargetSupport`. -- The pipeline is a **fixed canonical pipeline** (LLVM-style), not adaptive request-driven ordering. -- **Inlining is a VIR/IPA transform** (HIR only requests it). -- `RangeInfo` is **lightweight**, seeded by `Range`/`Assume` traits — no full ScalarEvolution. -- Analysis invalidation is **fine-grained** (preserved-analyses sets). - ---- - -## 7. C Backend (`vct.backend.c`) - -**Role:** VIR → C17. - -### 7.1 Contract - -- **Input:** optimized VIR (still SSA, traits attached). -- **Output:** one C17 translation unit **per function**. -- Emission is per-function independent → parallel; the driver concatenates deterministically. -- Two-pass emission: forward declarations, then definitions. - -### 7.2 Out-of-SSA - -Boissinot et al., *"Revisiting Out-of-SSA Translation"* — handles the lost-copy and swap problems. **Critical edges are split before** out-of-SSA. - -### 7.3 Compiler-friendly C - -- **Structured control-flow reconstruction** (`if`/`else`/`while`/`for`/`do`/`switch`) for reducible CFGs; `goto` fallback for irreducible. -- **Qualifiers from traits:** `restrict` (`Restrict`/`NoAlias`), `const` (`ReadOnly`/`NoWrite`), `_Noreturn` (`NoReturn`), `cold`/`hot` (`ColdPath`/`HotPath`), `static inline` / `noinline`. -- **Hints:** `__builtin_expect`; `__builtin_assume_aligned` / `_Alignas` (`Align`); `__builtin_unreachable` (`Unreachable`); `__builtin_assume` (clang, `Assume`); `#pragma GCC ivdep` / `#pragma clang loop vectorize(enable)` (`Vectorize`); `pure`/`const` (`NoRead`/`NoWrite`). -- **`NoOptimize`/`complex` barrier:** a compiler fence — `__asm__ __volatile__("" ::: "memory")` plus an opaque `noinline` call. - -### 7.4 Types & layout - -`intN_t`/`uintN_t`, `float`/`double`, `T*`, `struct`/`union`/arrays, function pointers, `void`. VIR aggregate GEPs use **field indices**, not byte offsets; the C compiler picks layout. An explicit **layout trait** pins the ABI (emits padding + `_Static_assert`). - -### 7.5 Naming - -Deterministic mangling. Exported names preserved; internal names sanitized and uniquified. A reserved-word/collision prefix table. - -### 7.6 Intrinsics - -Mostly `__builtin_*` (GCC/Clang), plus a small set of portable fallback helpers for what builtins do not cover. - -### 7.7 Debug mapping - -`#line` directives back to frontend source, flag-gated (on for debug builds, off for release). The frontend registers its source map with `vct.diag`. - -### 7.8 ABI - -Exported functions match the C ABI; internal functions are `static`; struct-by-value follows the C compiler's ABI. `MustTail` requires `[[clang::musttail]]` on Clang; on GCC a `MustTail` request is a **hard error**. - ---- - -## 8. LLVM Backend (`vct.backend.llvm`) - -**Role:** VIR → textual LLVM IR, **post** VIR optimization. Pure translation; no optimization or discovery. Output is a self-contained `.ll` module. libVCT never invokes `llvm-as`/`opt`/`llc` or links LLVM — that is the frontend's job. Deterministic per-function emission with stable concatenation. - -### 8.1 SSA mapping - -VIR is already SSA → **1:1 mapping**: instruction → instruction, block → block, `Phi` → `phi`, terminators, `alloca`/`load`/`store`. **No out-of-SSA** (the opposite of the C backend). - -### 8.2 Types - -`iN` integers, `float`/`double`, **opaque `ptr`** (not typed), `struct`/array (VIR field indices → LLVM struct indices), `void`, function types. - -### 8.3 Traits → LLVM attributes/metadata - -| Trait | LLVM | -|---|---| -| `Restrict`/`NoAlias` | `noalias` + `!alias.scope`/`!noalias` | -| `ReadOnly`/`WriteOnly`/`NoRead`/`NoWrite` | `readonly`/`writeonly` | -| `NonNull` | `nonnull` | -| `Align(n)` | `align n` | -| `Range` | `!range` on loads | -| `Assume` | `llvm.assume` | -| `NoInline`/`AlwaysInline` | `noinline`/`alwaysinline` | -| `TailCall`/`MustTail`/`NoTail` | `tail`/`musttail`/`none` | -| `NoReturn` | `noreturn` | -| `HotPath`/`ColdPath` | `!prof` | -| `Vectorize` | `!llvm.loop.vectorize.enable` | -| `Unroll(n)` | `!llvm.loop.unroll.count` | -| `Unreachable` | `unreachable` | -| `NoUndef` | reserved; only meaningful here | - -### 8.4 Intrinsics - -`memcpy`/`memset`/`memmove`, `llvm.sqrt.*`, `llvm.fabs.*`, `llvm.ctpop.*`, `llvm.fshl.*`, saturating/overflow ops. - -### 8.5 Module scaffolding - -Target triple (from `-target`) + derived target datalayout, so the module is self-contained. Declarations precede definitions. - -### 8.6 Resolutions - -- Pin **LLVM 18+** only (opaque pointers, current metadata). -- **No `!llvm.dbg`** in v1; debug only on the C path via `#line`. -- Emit **text `.ll` only**; the caller runs `llvm-as`. -- Emit **real** `llvm.assume` / `!range`. - ---- - -## 9. Driver & CLI - -### 9.1 Role split - -- **`vct.driver`** — library-level orchestrator: takes emitted C TUs, invokes the C compiler, collects `.o`. Owns the `-llvm` path (returns LLVM IR, never touches `cc`) and the `-mangled` path. -- **`vct.cli`** — thin wrapper: parses flags into `Config`, drives the whole pipeline (read IR → HIR opt → lower → VIR opt → `backend.c` → driver), formats diagnostics, sets exit codes. GC-allocated; holds no optimizer state. - -### 9.2 Pipeline - -``` -read input - -> (HIR opt if enabled) - -> lower - -> (VIR opt if enabled) - -> backend.c -> driver -> .o -``` - -`-S` stops after C; `-emit-llvm`/`-llvm` stop after LLVM IR. - -### 9.3 Flag surface (v1) - -| Group | Flags | -|---|---| -| **Optimize** | `-O0` `-O1` `-O2` `-O3` `-Ofast` `-Oz` | -| **Target** | `-march=` (→ cflags), `-mcpu=`, `-target ` | -| **Modes** | `-S` (C only), `-emit-llvm`/`-llvm`, `-mangled` (cosmopolitan) | -| **Toolchain** | `-cc=` (default auto), `-cflags="..."`, `-j`, `-save-temps` | -| **LTO** | `-flto[=full\|thin]`, `-ffat-lto-objects` | -| **Debug** | `-g`, `--dump-hir`, `--dump-vir`, `--verify`, `--time-passes`, `--stats` | -| **Output** | `-o ` | -| **Warnings** | `-Wall -Werror -Wextra` **always on**; `-Wno-error` escape hatch; `-w` to silence | -| **Diagnostics** | `--remarks`, `--diagnostics=json` | -| **Pass toggles** | every pass exposed as `-f` / `-fno-` | - -### 9.4 cc invocation - -``` -cc -std=c17 -c -o -Wall -Werror -Wextra -``` - -One process per TU, bounded by `-j`. The compiler is discovered via `$CC` then `PATH`; its version/dialect is probed once and drives the backend's pragma/builtin choices (§7.3, §7.6). - -### 9.5 Error handling - -`cc` stderr is captured. `#line` directives (§7.7) map C errors back to VIR/source; they are re-emitted via `vct.diag`. A `cc` failure produces a diagnostic and a nonzero exit. - -### 9.6 Determinism - -Stable TU ordering and flag ordering. Temp files live under a GC-managed temp dir (cleaned unless `-save-temps`). - -### 9.7 `-mangled` - -Switches the toolchain to `cosmocc`/cosmopolitan libc and produces an Actually Portable Executable. This mode **does link** — the opt-in exception to "caller links". - -### 9.8 Resolutions - -- Driver compiles to `.o` only; a single `-o` merges per-function objects with an `ld -r` relocatable link; a recommended link line / response file is emitted for the caller. `-mangled` is the sole mode producing a final executable. -- The CLI also reads a **textual HIR/VIR format** for testing/tools/round-trips; frontends still embed the library. -- **Every pass** is exposed as a `-f`/`-fno` toggle. -- `-Wall -Werror -Wextra` default; `-Wno-error` escape hatch; `-Werror` applies to libVCT-generated C. - ---- - -## 10. Diagnostics & Source Maps (`vct.diag`) - -### 10.1 Central service - -One central diagnostics service; every module emits through it, nothing prints directly. GC-allocated. - -```d -struct Diagnostic { - Severity severity; // Error | Warning | Note | Remark | Ice - DiagCode code; // stable namespaced, e.g. VCT1002 - string message; - Span primary; - Span[] notes; - Suggestion[] suggestions; -} -``` - -### 10.2 Source maps - -The frontend registers `SourceLocation{file,line,col,len}` against HIR nodes; VIR resolves via the `LoweringMap` (§5.6). Degradation to no-location is graceful. The C backend emits `#line` (§7.7); `cc` errors are parsed and re-mapped (§9.5). - -### 10.3 Channels - -- `DiagnosticConsumer` callback (canonical for embedded use). -- Human-readable CLI (color when tty). -- JSON (`--diagnostics=json`, **versioned schema**, shipped in v1). - -### 10.4 Optimizer remarks - -`--remarks` (off by default; automatically enabled at `-O3`) emits per-decision remarks tied to the trait pipeline, e.g.: - -- `HIR: replaced 3 uses of node#412 (comptime 64)` -- `VIR: denied Inline on @foo — CostModel (callee 2.4x size budget)` - -### 10.5 ICE policy - -Internal compiler errors include: missing/contradictory trait at lowering (§5.1), VIR invariant violated in debug (§5.10), and a Strong request denied for `Illegality`/`ContradictsTrait` (§6.8). - -- **Release:** return a failure result + ICE diagnostic with a module-dump request. -- **Debug:** assert. - -Never silently continue. - -### 10.6 Exit codes - -| Code | Meaning | -|---|---| -| 0 | success | -| 1 | diagnostics present | -| 2 | usage/config error | -| 3 | ICE | - -### 10.7 Resolutions - -- The trait-mismatch warning **is** promoted to error under `-Werror` (frontends can opt out with `-Wno-error=`). -- ICE in release = diagnostic + error result. -- Remarks: both opt-in `--remarks` **and** automatically enabled at `-O3`. -- Ship a versioned JSON diagnostics format in v1. - ---- - -## 11. Testing & Verification - -### 11.1 Core constraint - -VIR is a strict trait consumer and cannot be tested without a valid HIR fixture. The test strategy is anchored on a fixture layer. - -### 11.2 Layers - -1. **Unit tests** — D `unittest` per module. -2. **HIR test-builder harness** (`vct.test.hirbuild`) — a fluent builder constructing HIR modules + traits without a frontend. This is the fixture layer for HIR-opt, lowering, VIR-opt, and both backends. -3. **Textual round-trip** — property: `print(parse(print(m))) == print(m)`. -4. **FileCheck-style tests** — `CHECK` / `CHECK-NOT` / `CHECK-NEXT` directives. -5. **Verifier** — structural SSA (§5.10) + trait consistency, via `--verify`; asserted after every pass in debug. -6. **Differential** — the VIR reference interpreter vs the compiled `.o`; the C path vs the LLVM path. -7. **Fuzz** — random HIR/trait combinations; property: terminates, verifier clean, output compiles. -8. **Matrix** — gcc + clang × `-O0`..`-Ofast`/`-Oz` × C/LLVM. -9. **Perf benchmarks** — codegen quality is the headline; compile time tracked via `--time-passes`/`--stats`. - -### 11.3 VIR reference interpreter - -Approved as **v1 scope**. It is the differential oracle: the same VIR is run through the interpreter and through the compiled `.o`, and results are compared. - -### 11.4 Determinism - -Every pipeline run is executed twice in CI; output must be byte-identical. - -### 11.5 Regression corpus - -A corpus of input IR + expected FileCheck patterns; CI runs the full matrix. - -### 11.6 Resolutions - -- FileCheck-style matcher (not exact golden files). -- Build the VIR reference interpreter in v1 as a differential oracle. -- Verifier always-on in debug builds; `--verify` opt-in in release. -- Random-HIR + random-trait fuzzers in v1. - ---- - -## 12. Build, Packaging & Public API Lifecycle - -### 12.1 Language & toolchain - -Written in D. **LDC2 is used for release builds** (best codegen); **DMD is used for debug builds** (faster compile/iteration). **Zero runtime dependencies** — only a D compiler to build, and a system C compiler discovered at runtime for the C path. - -### 12.2 Build system - -**`xmake`** is the build system — not `dub`. A single `xmake.lua` drives all targets: the release profile uses LDC2, the debug profile uses DMD. No external package manager is required to build. - -### 12.3 Artifacts - -- `libvct.a` (static) and `libvct.so` (shared). -- `vct` CLI binary. -- `vctc.h` — C API header. -- `vct.test.hirbuild` — shipped publicly. - -### 12.4 API surfaces - -- **Thin `extern(C)` C API** (v1, the primary public surface): opaque handles for `Context`/`Module`/`Builder`/`Config`, functions to build IR, run the pipeline, and query diagnostics. No logic lives in the shim. -- **Native D API** (full feature set): the implementation surface; the C API is a thin shim over it, so the D API is exercised by every C API call. - -### 12.5 Module visibility - -- **Public:** `vct.ir.hir`, `vct.traits`, `vct.context`, `vct.backend.c` entry, `vct.backend.llvm` entry, `vct.driver`, `vct.diag`, `vct.test.hirbuild`. -- **Internal:** `vct.ir.vir`, `vct.lower`, `vct.hir.opt`, `vct.vir.opt`, `vct.comptime` internals. - -### 12.6 Memory ownership & lifecycle - -- A `Context` owns one or more **arenas**; an arena is the unit of reclamation. Default: one arena per `Module`, freed wholesale. -- All IR and optimizer objects are arena-owned and non-GC; passes mutate in place under epoch guards. -- GC is permitted only on cold paths (driver, diagnostics, CLI). -- **Arena hooks are exposed**: `Context` accepts an **`ArenaAllocator` interface** (`allocate`/`reset`/`destroy`) so embedders can supply their own backing memory; a default bump allocator ships in-tree. - -### 12.7 `Config` - -One struct holds opt level, target (`-march`/`-mcpu`/`-target`), all budgets, pass toggles, warning policy, cc selection, and mode (`-S`/`-llvm`/`-mangled`). The CLI is a pure function from argv to `Config`; the library takes a `Config`. Programmatic callers bypass the CLI. - -### 12.8 Threading contract - -Thread-safe when each compilation unit has its own `Context`/arenas; no shared mutable global state. Intra-module parallelism is internal and bounded by `-j`. - -### 12.9 Versioning - -- Library semver. -- **Textual IR format version** — separate and independently versioned. -- **Trait vocabulary version** — adding attributes/requests is backward-compatible; changing semantics is a major bump. Experimental passes are gated behind feature flags. - -### 12.10 Determinism guarantee - -Same `Config` + same input ⇒ byte-identical output, independent of thread count. - -### 12.11 Resolutions - -- Thin C API first (the primary public surface in v1). -- `ArenaAllocator` hooks exposed. -- LDC2 for release builds; DMD for debug builds. -- `xmake` as the build system (not `dub`). - ---- - -## 13. Worked End-to-End Example - -This example is **normative** for trait/request semantics. - -Input program: - -``` -fn foo(x: int, y: int) -> int { - return x + y; -} - -fn main() -> int { - x = 4; - y = x + 4; - z = foo(x, y); - a = sqrt(z); - println(a); -} -``` - -The HIR cascade proceeds as follows: - -1. HIR comptime-evaluates `x = 4` → `is_comptime`, `is_constant`, `const_value = 4`. -2. `y = x + 4` unfolds to `y = 8` → `is_comptime`. -3. `foo(x, y)` is proven small → HIR issues an `Inline` **request**; after inlining, `z = 4 + 8 = 12` → comptime. -4. `a = sqrt(z)` → `a = sqrt(12)` comptime → `a = 3.4641016151377544`. -5. The whole program collapses to `println(3.4641016151377544)`. - -VIR then optimizes the `println` call. The emitted C is effectively a single call with the folded constant. This demonstrates trait propagation HIR → VIR and the request lifecycle (`Inline` accepted). - ---- - -## 14. Open Questions & Future Work - -- **Debug info on the LLVM path** (`!llvm.dbg`) — deferred past v1. -- **Windows / MSVC support** — out of scope for v1. -- **LLVM bitcode emission** — deferred; caller runs `llvm-as`. -- **Incremental / cached compilation** — content-hash `.o` caching is out of scope for v1. -- **Cross-compilation targets** beyond `-target` passthrough and cosmopolitan — future work.