# libVCT — Vox C Transpiler Library: Architecture Specification - **Status:** Draft (pending final review) - **Date:** 2026-09-13 - **Scope:** Whole-library architecture — component boundaries and interfaces. Subsystem internals are deferred to later specs. - **Normative language:** `MUST`, `SHOULD`, `MAY` per RFC 2119. --- ## 1. Overview libVCT is an LLVM-inspired compiler library written in D. It consumes an IR, optimizes it, and transpiles it into C17, then drives a system C compiler to produce an object file. An alternate backend emits textual LLVM IR instead of C. The library is organized around a **two-stage IR**: - **HIR** — a high-level, tree-shaped AST that is the frontend's output and is **annotated in place** with *traits*. HIR does discovery and small, cheap folds. - **VIR** — a low-level SSA/CFG instruction IR. VIR does the heavy lifting and performs **no discovery**; it is a strict consumer of HIR-provided traits. The design thesis: because HIR hands VIR a fully annotated "dictionary" of facts, VIR can skip analysis and go straight to transformation. HIR's cheap, cascading folds are parallelizable; the resulting VIR is simpler and faster to optimize. The net goal is LLVM-class codegen quality with materially lower compile time. ### 1.1 Goals 1. **Codegen quality is the headline.** Generated code quality takes priority over compile speed. 2. **Two-stage optimization.** HIR performs small folds and trait derivation; VIR performs large transformations. 3. **Trait-driven VIR.** VIR never re-analyzes; every fact it uses is an attribute, relationship, or request supplied by HIR. 4. **Parallel, cascading HIR.** HIR passes run per-function in parallel and cascade to a fixpoint. 5. **Compiler-friendly C.** The C backend always emits C written to be pattern-matched by GCC/Clang. 6. **Deterministic output.** Same input + same config ⇒ byte-identical output, regardless of thread count. ### 1.2 Non-goals (v1) - **Linking.** libVCT produces object files; linking is the caller's job. The sole exception is `-mangled` (cosmopolitan) mode. - **Windows / MSVC.** Linux/POSIX-first; gcc and clang only. - **LLVM bitcode.** The LLVM backend emits textual `.ll` only; the caller runs `llvm-as`. - **LLVM debug metadata.** No `!llvm.dbg` in v1; debug mapping exists on the C path via `#line`. - **A hand-authored VIR.** VIR is internal and intentionally hostile to hand-authoring. A textual form exists solely for tools and round-trip tests (§9.8); it is not a supported authoring surface. - **Consuming LLVM IR or Vox's existing IR.** libVCT owns its IR definition. ### 1.3 Glossary | Term | Meaning | |---|---| | **HIR** | High-level IR: the frontend AST, annotated in place with traits. | | **VIR** | Vox IR: a low-level SSA/CFG instruction IR. | | **Trait** | The union of Attribute, Request, and Relationship attached to a node. | | **Attribute** | A fact about a node (type, value, mutability, ...). | | **Request** | An attribute-derived suggestion from HIR to VIR. | | **Relationship** | A directed edge between nodes, traceable up/down. | | **Cascade** | The chain of HIR rewrites triggered by one fold. | | **Lowering** | The "middleman" that translates annotated HIR into VIR SSA. | | **Remark** | An optimizer decision record emitted for diagnostics. | --- ## 2. Component Architecture ### 2.1 Modules | Module | Responsibility | Allocation | Visibility | |---|---|---|---| | `vct.ir.hir` | HIR AST node types, Trait model, builder/emitter API | arena | **public** | | `vct.ir.vir` | VIR SSA/CFG types, VIR type system, instruction set | arena | internal | | `vct.traits` | Attribute / Request / Relationship definitions + queries | arena | **public** | | `vct.comptime` | Comptime evaluator / constant-folding engine | arena | internal | | `vct.hir.opt` | HIR pass framework + passes, parallel cascade | arena | internal | | `vct.lower` | HIR → VIR lowering + SSA construction | arena | internal | | `vct.vir.opt` | VIR pass framework + heavy passes | arena | internal | | `vct.backend.c` | VIR → C17 emitter (out-of-SSA, compiler-friendly C) | arena + GC | **public entry** | | `vct.backend.llvm` | VIR → textual LLVM IR emitter | arena + GC | **public entry** | | `vct.driver` | cc invocation, flag assembly, `.o` emission, LTO, cosmopolitan | GC | **public** | | `vct.cli` | Thin command-line wrapper | GC | **public** | | `vct.diag` | Diagnostics, source maps, error reporting | GC | **public** | | `vct.context` | Context / arena ownership, `Config` | GC + arena | **public** | | `vct.test.hirbuild` | Fluent HIR test-builder harness | arena | **public** | ### 2.2 Data flow ``` frontend --build--> HIR --hir.opt (parallel, cascading)--> annotated HIR --lower--> VIR (SSA/CFG) --vir.opt--> optimized VIR --backend.c--> C17 source --driver--> .o (default path) optimized VIR --backend.llvm--> textual LLVM IR (caller links LLVM) ``` ### 2.3 Public interfaces 1. **HIR builder (frontend-facing).** Construct nodes, set types/values, attach frontend-supplied traits (`static`, hints), and finish a module. 2. **Trait query (VIR-facing).** Read attributes and walk relationships. This is the **only** channel through which VIR obtains facts. 3. **Backend (driver-facing).** Consume optimized VIR and emit C (or textual LLVM IR). ### 2.4 Load-bearing boundaries - **`vct.traits` is the contract.** Both HIR and VIR depend on it. VIR depends on nothing else from HIR except **node identity**. - **`vct.lower` is the SSA construction site.** Phi-nodes, dominance, and the memory model all land here; it is the most algorithmically dense module. --- ## 3. The Trait Model A `Trait` is the union of three sub-structures hung on HIR nodes, and the entire vocabulary VIR may reason with: ```d struct Trait { Attribute attr; Request[] reqs; Relation[] rels; } ``` Traits are **exposed publicly**. A frontend MAY **suggest** traits to HIR; HIR validates suggestions and fills in correct values. Derived traits are authoritative and VIR trusts them blindly. ### 3.1 Attributes Attributes are facts about a node, with **two origins in one namespace**: - **Suggested** — set by the frontend at build time. HIR **validates**; on mismatch it emits a warning (promoted to error under `-Werror`) and **overwrites** with the correct value. - **Derived** — produced by the HIR optimizer during the cascade. Authoritative. Every attribute carries an `attr_source` bit recording whether it was suggested or derived. **v1 attribute set:** | Attribute | Meaning | |---|---| | `ty` | The node's VIR type. | | `const_value` | Present iff the value is known at compile time. | | `is_static` | Frontend-declared compile-time value. | | `is_comptime` | HIR-proven compile-time value. | | `is_constant` | Value is constant (not necessarily comptime). | | `is_used` | Node has at least one use. | | `is_mutably_used` | Node is mutated through at least one use. | | `is_addressed` | Address taken by pointer/reference. | | `escapes` | Value escapes its defining scope. | | `is_runtime_mutable` | May change at runtime. | | `may_change_at_runtime` | The observable value may differ between reads/executions (external state). Distinct from `is_runtime_mutable`, which means the storage is written at runtime. | | `complex` | Frontend-provided; gates the `NoOptimize` request. | ### 3.2 Requests Requests are **attribute-derived** suggestions from HIR to VIR — never ad-hoc. Each has a **strength**: - `Soft` — a suggestion VIR MAY decline. - `Strong` — violating it is likely a bug (e.g., `NoInline`, `MustTail`, `AlwaysInline`, `NoOptimize`). Every request produces a `RequestResult`: ```d struct RequestResult { Request req; bool accepted; DenyReason reason; string note; // one-line human-readable explanation } ``` A denial MUST carry a `DenyReason` enum (for tooling) and a one-line `note`. ```d enum DenyReason { CostModel, // profitable only under a different cost model Illegality, // transform would be incorrect AlreadyDone, // no-op: the goal already holds Unsupported, // VIR does not implement it yet ContradictsTrait, // conflicts with a stronger fact TooLarge, // exceeds a size budget NoTargetSupport, // backend/target cannot express it } ``` #### 3.2.1 Request catalog (v1) **Inlining / call edges:** `Inline`; `AlwaysInline` (Strong); `NoInline` (Strong); `TailCall`; `MustTail` (Strong); `NoTail` (Strong); `Devirtualize`; `ColdCall`; `LikelyCall`; `UnlikelyCall`. **Loops:** `Vectorize`; `Unroll(factor)`; `Interleave(count)`; `Peel(count)`; `Distribute`; `Fuse`; `Jam`; `Unswitch`; `Rotate`; `LICM` (action); `LoopInvariant` (property); `MustProgress`. **Memory effects:** `NoAlias`; `Restrict`; `NonNull`; `Align(n)`; `ReadOnly`; `WriteOnly`; `NoRead`; `NoWrite`; `NoCapture`; `Dereferenceable(n)`; `Constant`; `NoUndef` (reserved); `Prefetch`. **Control flow:** `HotPath`; `ColdPath`; `Unreachable`; `NoReturn`. **Assumptions:** `Assume(pred)`; `Range(lo, hi)`. **Optimization control:** `NoOptimize` (Strong; requested **only** when the frontend sets `complex` on the node). #### 3.2.2 Request conflict rules - `Strong` beats `Soft`. - Two conflicting `Strong` requests (e.g., `Inline` vs `NoInline`, `TailCall` vs `NoTail`) ⇒ HIR emits a diagnostic and **rejects** the frontend suggestion. - `LoopInvariant` (a proven property VIR may exploit) and `LICM` (an explicit request to hoist) are **distinct** and both retained. - `NoRead`/`NoWrite` are **primitives on an access edge**; `ReadOnly`/`WriteOnly` are **region-level facts** derived from them. All four are retained, with the scope difference documented. ### 3.3 Relationships Relationships are directed edges HIR leaves for the non-constant world. They are traceable **up and down** to a terminal node. **Flags:** | Flag | Meaning | |---|---| | `is_offspring` | Cannot trace up. | | `is_ancestor` | Cannot trace down. | | `is_common` | Branch point; VIR must choose a direction. | | `has_siblings` | Connects down to 2+ nodes. | | `is_apex` | Part of a cycle (`a → b → c → d → a`), usually from polymorphism. | **Relation attributes (small set):** `was_changed`; `is_mutable`; `is_pointer`. An apex SHOULD normally be ironed out by HIR; residual apexes typically arise from polymorphism. ### 3.4 Representation & staleness - **Representation (resolved):** an **embedded dense `Trait` struct** on every node is primary, plus an **optional sparse side-channel** for rare / frontend-extensible attributes. - **Staleness (resolved):** **epoch/generation counters** with dirty propagation along relationships. A trait is valid only for the node's current epoch. This survives parallel HIR passes. --- ## 4. HIR Optimizer **Role:** discovery + small folds + trait derivation. HIR hands VIR a "dictionary". ### 4.1 Pass framework ```d interface HirPass { string name(); void run(HirFunction fn, HirContext ctx); } ``` Passes mutate the annotated AST in place. Successful rewrites emit **triggers**: ```d enum Trigger { ConstantUnfolded, UsesReplaced, NodeDeleted, TraitChanged, StaticDiscovered, } ``` A **worklist scheduler** collects `(pass, node)` pairs, dedupes, and runs to a fixpoint. A per-function step budget guarantees termination. ### 4.2 The cascade Canonical chain: ``` 4 * 16 -> const 64 -> mark is_comptime / is_constant / const_value = 64 -> prove not addressed / not runtime-mutable -> replace all uses with literal 64 -> node dead -> delete -> downstream nodes now constant -> enqueue ``` Each hop emits a trigger; the worklist drains until nothing new is provable. **HIR never inlines** — it only proves smallness and issues an `Inline` request to VIR. ### 4.3 Parallelism & determinism - Parallelize **per function** (independent HIR trees). - Within a function, passes run **sequentially** so cascade order is deterministic. - Cross-function effects (inlining, global constant propagation) use a **module-level fixpoint**: parallel per-function passes → deterministic module phase → re-enqueue only affected functions. - Epoch counters prevent stale trait reads in parallel workers. ### 4.4 Comptime evaluator An interpreter over HIR subgraphs. Eligible only if: no side effects, primitives only, no global mutation, no I/O, and within loop/recursion budgets. Otherwise it bails and defers to VIR. Outputs `const_value`, then `is_comptime` + `is_constant`. The frontend `static` attribute **seeds** the evaluator. HIR trusts `static` as "this is compile-time" but still validates it. **Budgets (configurable; defaults):** | Budget | Default | |---|---| | `comptime.max_iterations` | 4096 | | `comptime.max_recursion_depth` | 256 | | `comptime.max_steps` | 1,000,000 | | `comptime.max_aggregate_elements` | 65,536 | | `hir.max_rewrites_per_function` | 100,000 | | `hir.max_rewrites_per_module` | 1,000,000 | All are overridable via the API and CLI. ### 4.5 Defer HIR carries a real high-level `defer` statement. The optimizer inlines the deferred call at every scope-exit path and then erases the `defer` marker — zero runtime tax. ### 4.6 Optimization-level behavior | Level | HIR | VIR | |---|---|---| | `-O0` | traits validated only | off | | `-O1` | builtin folds + discovery | off | | `-O2` | full cascade + comptime | **on** | | `-O3` | + special AST transforms (loop-unfold/vectorize → constant-unfold → comptime) | on | | `-Ofast` | = `-O3` + `-march=native` + fast-math | on | | `-Oz` | size-tuned | on | **Compiler-friendly C is emitted at ALL optimization levels.** `-O3` merely adds more aggressive AST transforms. --- ## 5. Lowering (HIR → VIR) **Role:** the "middleman". Pure translation + SSA construction; **no discovery**. ### 5.1 Contract - **Input:** annotated HIR with **complete, epoch-valid** traits. - **Output:** VIR — basic blocks, phi-nodes, def-use, SSA. - Lowering **validates trait completeness**. A missing or contradictory trait raises an **internal-compiler-error** diagnostic (see §10.5). - Deterministic; parallelizable per function. ### 5.2 Form Lowering is a **streaming recursive interpreter** over the HIR tree ("tcc-for-VIR"): - Structured HIR (`if`/`while`/`for`/`switch`/`block`) → blocks + terminators (`br`/`condbr`/`switch`/`ret`/`unreachable`). - Expressions → temporaries + instructions. ### 5.3 SSA construction SSA is built with the **Braun et al. sealed-block algorithm**: one pass, on-the-fly phi insertion, no separate dominance-frontier pass. A loop header is sealed once its back-edge is emitted. ### 5.4 Memory model (trait-driven hybrid) | Condition | Representation | |---|---| | scalar ∧ ¬`is_addressed` ∧ ¬`escapes` ∧ ¬`is_runtime_mutable` | pure SSA value | | addressed / escaping / runtime-mutable / aggregate | `alloca` + explicit `load`/`store` | | volatile / atomic | forced memory, never promoted | VIR's `mem2reg`/`SROA` MAY promote memory back to SSA when traits confirm safety. ### 5.5 Phi VIR has an explicit LLVM-style `Phi` instruction whose operands are `(value, predecessor-block)` pairs. ### 5.6 Identity & traits A `LoweringMap` (HIR node → VIR entity(ies)) is built during lowering. Each VIR entity carries its own `Trait`, populated from the source HIR node. HIR Relationship endpoints are rewritten to VIR entities via the map. VIR depends only on `vct.traits` + node identity. ### 5.7 Optimization barriers `complex`/`NoOptimize` nodes are lowered inside a **region-level** optimization barrier. VIR passes MUST NOT rewrite across it. ### 5.8 Defer fallback If HIR optimization is off (`-O0`/`-O1`), lowering itself expands `defer` at every scope exit. ### 5.9 VIR type system More minimal than HIR but still fleshed out: integers with width, floats, pointers, aggregates, function types, `void`. ### 5.10 Output invariants Well-formed SSA: - every use is dominated by its def, - every block is terminated, - phi arity equals predecessor count, - a single def per value. Verified in debug builds (§11). --- ## 6. VIR Optimizer **Role:** the heavy-lifting half. Reads traits only, transforms SSA, performs **no discovery**. Runs at `-O2`/`-O3`/`-Ofast`/`-Oz`. ### 6.1 Contract - **Input:** lowered VIR (well-formed SSA/CFG) + complete traits. - **Output:** optimized VIR, still in SSA. - **Strict trait consumer:** a transform is enabled by an attribute/relationship or requested by a Request. VIR MUST NOT assume a fact absent from traits. ### 6.2 Framework ```d interface VirPass { string name(); void run(VirModule m, PassContext ctx); } ``` **Analyses:** `DominatorTree`, `PostDominatorTree`, `LoopInfo`, `AliasInfo`, `DefUse`, `CallGraph`, `RangeInfo`. Passes declare which analyses they preserve; the manager invalidates the rest (**fine-grained**, not invalidate-all). Epoch counters catch stale trait reads. ### 6.3 Default pipeline (`-O2`) A module pass manager interleaving function passes and IPA: 1. **Canonicalize:** `mem2reg`, `SROA`, `instcombine`, `simplifyCFG`, `early-CSE`, `DCE`. 2. **Scalar:** `GVN`, `SCCP`, `LICM`, `indvars`, `reassociation`. 3. **IPA:** inliner (driven by `Inline`/`AlwaysInline`/`NoInline`/`ColdCall`/`LikelyCall`), global `DCE`, `IPSCCP`, function-attribute propagation. 4. **Loops:** `unroll`/`interleave`/`peel`/`rotate`/`unswitch`/`distribute`/`fuse`/`jam`, each gated by its Request. 5. **Vectorize:** loop + SLP, gated by `Vectorize`/`LoopInvariant`/`Range`/`NoAlias`/`Restrict`. 6. **Memory:** alias-driven `DSE`, GEP simplification, load widening. 7. **Control flow:** block layout (`HotPath`/`ColdPath`), tail-call formation (`TailCall`/`MustTail`/`NoTail`), unreachable pruning, jump threading. 8. **Codegen prep:** canonicalize for the backend; remains SSA. `-O3`/`-Ofast` raise aggression. `-Oz` is size-first (most unrolling/vectorization off). ### 6.4 Requests Each Request yields a `RequestResult` (§3.2). Denials carry a `DenyReason` + one-line note. ### 6.5 Alias analysis Synthesized from `NoAlias`/`Restrict`/`NoCapture`/`Dereferenceable`/`ReadOnly`/`WriteOnly` + provenance — no guessing. ### 6.6 Barriers Region optimization barriers (`complex`/`NoOptimize`, §5.7) are opaque; passes skip across them. ### 6.7 Budgets `vir.max_iterations`, `vir.max_pipeline_rounds` (configurable). ### 6.8 Resolutions - A **Strong-request denial is a hard error** when the backend is capable and the reason is `Illegality`/`ContradictsTrait`; it is a **warning** for `Unsupported`/`NoTargetSupport`. - The pipeline is a **fixed canonical pipeline** (LLVM-style), not adaptive request-driven ordering. - **Inlining is a VIR/IPA transform** (HIR only requests it). - `RangeInfo` is **lightweight**, seeded by `Range`/`Assume` traits — no full ScalarEvolution. - Analysis invalidation is **fine-grained** (preserved-analyses sets). --- ## 7. C Backend (`vct.backend.c`) **Role:** VIR → C17. ### 7.1 Contract - **Input:** optimized VIR (still SSA, traits attached). - **Output:** one C17 translation unit **per function**. - Emission is per-function independent → parallel; the driver concatenates deterministically. - Two-pass emission: forward declarations, then definitions. ### 7.2 Out-of-SSA Boissinot et al., *"Revisiting Out-of-SSA Translation"* — handles the lost-copy and swap problems. **Critical edges are split before** out-of-SSA. ### 7.3 Compiler-friendly C - **Structured control-flow reconstruction** (`if`/`else`/`while`/`for`/`do`/`switch`) for reducible CFGs; `goto` fallback for irreducible. - **Qualifiers from traits:** `restrict` (`Restrict`/`NoAlias`), `const` (`ReadOnly`/`NoWrite`), `_Noreturn` (`NoReturn`), `cold`/`hot` (`ColdPath`/`HotPath`), `static inline` / `noinline`. - **Hints:** `__builtin_expect`; `__builtin_assume_aligned` / `_Alignas` (`Align`); `__builtin_unreachable` (`Unreachable`); `__builtin_assume` (clang, `Assume`); `#pragma GCC ivdep` / `#pragma clang loop vectorize(enable)` (`Vectorize`); `pure`/`const` (`NoRead`/`NoWrite`). - **`NoOptimize`/`complex` barrier:** a compiler fence — `__asm__ __volatile__("" ::: "memory")` plus an opaque `noinline` call. ### 7.4 Types & layout `intN_t`/`uintN_t`, `float`/`double`, `T*`, `struct`/`union`/arrays, function pointers, `void`. VIR aggregate GEPs use **field indices**, not byte offsets; the C compiler picks layout. An explicit **layout trait** pins the ABI (emits padding + `_Static_assert`). ### 7.5 Naming Deterministic mangling. Exported names preserved; internal names sanitized and uniquified. A reserved-word/collision prefix table. ### 7.6 Intrinsics Mostly `__builtin_*` (GCC/Clang), plus a small set of portable fallback helpers for what builtins do not cover. ### 7.7 Debug mapping `#line` directives back to frontend source, flag-gated (on for debug builds, off for release). The frontend registers its source map with `vct.diag`. ### 7.8 ABI Exported functions match the C ABI; internal functions are `static`; struct-by-value follows the C compiler's ABI. `MustTail` requires `[[clang::musttail]]` on Clang; on GCC a `MustTail` request is a **hard error**. --- ## 8. LLVM Backend (`vct.backend.llvm`) **Role:** VIR → textual LLVM IR, **post** VIR optimization. Pure translation; no optimization or discovery. Output is a self-contained `.ll` module. libVCT never invokes `llvm-as`/`opt`/`llc` or links LLVM — that is the frontend's job. Deterministic per-function emission with stable concatenation. ### 8.1 SSA mapping VIR is already SSA → **1:1 mapping**: instruction → instruction, block → block, `Phi` → `phi`, terminators, `alloca`/`load`/`store`. **No out-of-SSA** (the opposite of the C backend). ### 8.2 Types `iN` integers, `float`/`double`, **opaque `ptr`** (not typed), `struct`/array (VIR field indices → LLVM struct indices), `void`, function types. ### 8.3 Traits → LLVM attributes/metadata | Trait | LLVM | |---|---| | `Restrict`/`NoAlias` | `noalias` + `!alias.scope`/`!noalias` | | `ReadOnly`/`WriteOnly`/`NoRead`/`NoWrite` | `readonly`/`writeonly` | | `NonNull` | `nonnull` | | `Align(n)` | `align n` | | `Range` | `!range` on loads | | `Assume` | `llvm.assume` | | `NoInline`/`AlwaysInline` | `noinline`/`alwaysinline` | | `TailCall`/`MustTail`/`NoTail` | `tail`/`musttail`/`none` | | `NoReturn` | `noreturn` | | `HotPath`/`ColdPath` | `!prof` | | `Vectorize` | `!llvm.loop.vectorize.enable` | | `Unroll(n)` | `!llvm.loop.unroll.count` | | `Unreachable` | `unreachable` | | `NoUndef` | reserved; only meaningful here | ### 8.4 Intrinsics `memcpy`/`memset`/`memmove`, `llvm.sqrt.*`, `llvm.fabs.*`, `llvm.ctpop.*`, `llvm.fshl.*`, saturating/overflow ops. ### 8.5 Module scaffolding Target triple (from `-target`) + derived target datalayout, so the module is self-contained. Declarations precede definitions. ### 8.6 Resolutions - Pin **LLVM 18+** only (opaque pointers, current metadata). - **No `!llvm.dbg`** in v1; debug only on the C path via `#line`. - Emit **text `.ll` only**; the caller runs `llvm-as`. - Emit **real** `llvm.assume` / `!range`. --- ## 9. Driver & CLI ### 9.1 Role split - **`vct.driver`** — library-level orchestrator: takes emitted C TUs, invokes the C compiler, collects `.o`. Owns the `-llvm` path (returns LLVM IR, never touches `cc`) and the `-mangled` path. - **`vct.cli`** — thin wrapper: parses flags into `Config`, drives the whole pipeline (read IR → HIR opt → lower → VIR opt → `backend.c` → driver), formats diagnostics, sets exit codes. GC-allocated; holds no optimizer state. ### 9.2 Pipeline ``` read input -> (HIR opt if enabled) -> lower -> (VIR opt if enabled) -> backend.c -> driver -> .o ``` `-S` stops after C; `-emit-llvm`/`-llvm` stop after LLVM IR. ### 9.3 Flag surface (v1) | Group | Flags | |---|---| | **Optimize** | `-O0` `-O1` `-O2` `-O3` `-Ofast` `-Oz` | | **Target** | `-march=` (→ cflags), `-mcpu=`, `-target ` | | **Modes** | `-S` (C only), `-emit-llvm`/`-llvm`, `-mangled` (cosmopolitan) | | **Toolchain** | `-cc=` (default auto), `-cflags="..."`, `-j`, `-save-temps` | | **LTO** | `-flto[=full\|thin]`, `-ffat-lto-objects` | | **Debug** | `-g`, `--dump-hir`, `--dump-vir`, `--verify`, `--time-passes`, `--stats` | | **Output** | `-o ` | | **Warnings** | `-Wall -Werror -Wextra` **always on**; `-Wno-error` escape hatch; `-w` to silence | | **Diagnostics** | `--remarks`, `--diagnostics=json` | | **Pass toggles** | every pass exposed as `-f` / `-fno-` | ### 9.4 cc invocation ``` cc -std=c17 -c -o -Wall -Werror -Wextra ``` One process per TU, bounded by `-j`. The compiler is discovered via `$CC` then `PATH`; its version/dialect is probed once and drives the backend's pragma/builtin choices (§7.3, §7.6). ### 9.5 Error handling `cc` stderr is captured. `#line` directives (§7.7) map C errors back to VIR/source; they are re-emitted via `vct.diag`. A `cc` failure produces a diagnostic and a nonzero exit. ### 9.6 Determinism Stable TU ordering and flag ordering. Temp files live under a GC-managed temp dir (cleaned unless `-save-temps`). ### 9.7 `-mangled` Switches the toolchain to `cosmocc`/cosmopolitan libc and produces an Actually Portable Executable. This mode **does link** — the opt-in exception to "caller links". ### 9.8 Resolutions - Driver compiles to `.o` only; a single `-o` merges per-function objects with an `ld -r` relocatable link; a recommended link line / response file is emitted for the caller. `-mangled` is the sole mode producing a final executable. - The CLI also reads a **textual HIR/VIR format** for testing/tools/round-trips; frontends still embed the library. - **Every pass** is exposed as a `-f`/`-fno` toggle. - `-Wall -Werror -Wextra` default; `-Wno-error` escape hatch; `-Werror` applies to libVCT-generated C. --- ## 10. Diagnostics & Source Maps (`vct.diag`) ### 10.1 Central service One central diagnostics service; every module emits through it, nothing prints directly. GC-allocated. ```d struct Diagnostic { Severity severity; // Error | Warning | Note | Remark | Ice DiagCode code; // stable namespaced, e.g. VCT1002 string message; Span primary; Span[] notes; Suggestion[] suggestions; } ``` ### 10.2 Source maps The frontend registers `SourceLocation{file,line,col,len}` against HIR nodes; VIR resolves via the `LoweringMap` (§5.6). Degradation to no-location is graceful. The C backend emits `#line` (§7.7); `cc` errors are parsed and re-mapped (§9.5). ### 10.3 Channels - `DiagnosticConsumer` callback (canonical for embedded use). - Human-readable CLI (color when tty). - JSON (`--diagnostics=json`, **versioned schema**, shipped in v1). ### 10.4 Optimizer remarks `--remarks` (off by default; automatically enabled at `-O3`) emits per-decision remarks tied to the trait pipeline, e.g.: - `HIR: replaced 3 uses of node#412 (comptime 64)` - `VIR: denied Inline on @foo — CostModel (callee 2.4x size budget)` ### 10.5 ICE policy Internal compiler errors include: missing/contradictory trait at lowering (§5.1), VIR invariant violated in debug (§5.10), and a Strong request denied for `Illegality`/`ContradictsTrait` (§6.8). - **Release:** return a failure result + ICE diagnostic with a module-dump request. - **Debug:** assert. Never silently continue. ### 10.6 Exit codes | Code | Meaning | |---|---| | 0 | success | | 1 | diagnostics present | | 2 | usage/config error | | 3 | ICE | ### 10.7 Resolutions - The trait-mismatch warning **is** promoted to error under `-Werror` (frontends can opt out with `-Wno-error=`). - ICE in release = diagnostic + error result. - Remarks: both opt-in `--remarks` **and** automatically enabled at `-O3`. - Ship a versioned JSON diagnostics format in v1. --- ## 11. Testing & Verification ### 11.1 Core constraint VIR is a strict trait consumer and cannot be tested without a valid HIR fixture. The test strategy is anchored on a fixture layer. ### 11.2 Layers 1. **Unit tests** — D `unittest` per module. 2. **HIR test-builder harness** (`vct.test.hirbuild`) — a fluent builder constructing HIR modules + traits without a frontend. This is the fixture layer for HIR-opt, lowering, VIR-opt, and both backends. 3. **Textual round-trip** — property: `print(parse(print(m))) == print(m)`. 4. **FileCheck-style tests** — `CHECK` / `CHECK-NOT` / `CHECK-NEXT` directives. 5. **Verifier** — structural SSA (§5.10) + trait consistency, via `--verify`; asserted after every pass in debug. 6. **Differential** — the VIR reference interpreter vs the compiled `.o`; the C path vs the LLVM path. 7. **Fuzz** — random HIR/trait combinations; property: terminates, verifier clean, output compiles. 8. **Matrix** — gcc + clang × `-O0`..`-Ofast`/`-Oz` × C/LLVM. 9. **Perf benchmarks** — codegen quality is the headline; compile time tracked via `--time-passes`/`--stats`. ### 11.3 VIR reference interpreter Approved as **v1 scope**. It is the differential oracle: the same VIR is run through the interpreter and through the compiled `.o`, and results are compared. ### 11.4 Determinism Every pipeline run is executed twice in CI; output must be byte-identical. ### 11.5 Regression corpus A corpus of input IR + expected FileCheck patterns; CI runs the full matrix. ### 11.6 Resolutions - FileCheck-style matcher (not exact golden files). - Build the VIR reference interpreter in v1 as a differential oracle. - Verifier always-on in debug builds; `--verify` opt-in in release. - Random-HIR + random-trait fuzzers in v1. --- ## 12. Build, Packaging & Public API Lifecycle ### 12.1 Language & toolchain Written in D. Primary compiler: **LDC2**; **DMD** supported as a secondary build target. **Zero runtime dependencies** — only a D compiler to build, and a system C compiler discovered at runtime for the C path. ### 12.2 Build system `dub` for normal development, **plus** a plain `build.d`/Makefile path for no-dub builds. Both MUST produce identical artifacts. ### 12.3 Artifacts - `libvct.a` (static) and `libvct.so` (shared). - `vct` CLI binary. - `vctc.h` — C API header. - `vct.test.hirbuild` — shipped publicly. ### 12.4 API surfaces - **Native D API** (primary, full feature set). - **Thin `extern(C)` C API** (v1): opaque handles for `Context`/`Module`/`Builder`/`Config`, functions to build IR, run the pipeline, and query diagnostics. No logic lives in the shim. ### 12.5 Module visibility - **Public:** `vct.ir.hir`, `vct.traits`, `vct.context`, `vct.backend.c` entry, `vct.backend.llvm` entry, `vct.driver`, `vct.diag`, `vct.test.hirbuild`. - **Internal:** `vct.ir.vir`, `vct.lower`, `vct.hir.opt`, `vct.vir.opt`, `vct.comptime` internals. ### 12.6 Memory ownership & lifecycle - A `Context` owns one or more **arenas**; an arena is the unit of reclamation. Default: one arena per `Module`, freed wholesale. - All IR and optimizer objects are arena-owned and non-GC; passes mutate in place under epoch guards. - GC is permitted only on cold paths (driver, diagnostics, CLI). - `Context` accepts an **`ArenaAllocator` interface** so embedders can supply backing memory. ### 12.7 `Config` One struct holds opt level, target (`-march`/`-mcpu`/`-target`), all budgets, pass toggles, warning policy, cc selection, and mode (`-S`/`-llvm`/`-mangled`). The CLI is a pure function from argv to `Config`; the library takes a `Config`. Programmatic callers bypass the CLI. ### 12.8 Threading contract Thread-safe when each compilation unit has its own `Context`/arenas; no shared mutable global state. Intra-module parallelism is internal and bounded by `-j`. ### 12.9 Versioning - Library semver. - **Textual IR format version** — separate and independently versioned. - **Trait vocabulary version** — adding attributes/requests is backward-compatible; changing semantics is a major bump. Experimental passes are gated behind feature flags. ### 12.10 Determinism guarantee Same `Config` + same input ⇒ byte-identical output, independent of thread count. ### 12.11 Resolutions - C API in v1 (thin shim). - Custom `ArenaAllocator` hooks exposed. - LDC2 primary + DMD secondary. - `dub` + no-dub build path. --- ## 13. Worked End-to-End Example This example is **normative** for trait/request semantics. Input program: ``` fn foo(x: int, y: int) -> int { return x + y; } fn main() -> int { x = 4; y = x + 4; z = foo(x, y); a = sqrt(z); println(a); } ``` The HIR cascade proceeds as follows: 1. HIR comptime-evaluates `x = 4` → `is_comptime`, `is_constant`, `const_value = 4`. 2. `y = x + 4` unfolds to `y = 8` → `is_comptime`. 3. `foo(x, y)` is proven small → HIR issues an `Inline` **request**; after inlining, `z = 4 + 8 = 12` → comptime. 4. `a = sqrt(z)` → `a = sqrt(12)` comptime → `a = 3.4641016151377544`. 5. The whole program collapses to `println(3.4641016151377544)`. VIR then optimizes the `println` call. The emitted C is effectively a single call with the folded constant. This demonstrates trait propagation HIR → VIR and the request lifecycle (`Inline` accepted). --- ## 14. Open Questions & Future Work - **Debug info on the LLVM path** (`!llvm.dbg`) — deferred past v1. - **Windows / MSVC support** — out of scope for v1. - **LLVM bitcode emission** — deferred; caller runs `llvm-as`. - **Incremental / cached compilation** — content-hash `.o` caching is out of scope for v1. - **Cross-compilation targets** beyond `-target` passthrough and cosmopolitan — future work.