huntedbytheirs/libvct-spec
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
31c9450fc189d31e4c55895817b3c3baad588ce5
libvct-spec/docs/superpowers/specs/2026-09-13-libvct-design.md
T

33 KiB
Raw Blame History

libVCT — Vox C Transpiler Library: Architecture Specification

  • Status: Draft (pending final review)
  • Date: 2026-09-13
  • Scope: Whole-library architecture — component boundaries and interfaces. Subsystem internals are deferred to later specs.
  • Normative language: MUST, SHOULD, MAY per RFC 2119.

1. Overview

libVCT is an LLVM-inspired compiler library written in D. It consumes an IR, optimizes it, and transpiles it into C17, then drives a system C compiler to produce an object file. An alternate backend emits textual LLVM IR instead of C.

The library is organized around a two-stage IR:

  • HIR — a high-level, tree-shaped AST that is the frontend's output and is annotated in place with traits. HIR does discovery and small, cheap folds.
  • VIR — a low-level SSA/CFG instruction IR. VIR does the heavy lifting and performs no discovery; it is a strict consumer of HIR-provided traits.

The design thesis: because HIR hands VIR a fully annotated "dictionary" of facts, VIR can skip analysis and go straight to transformation. HIR's cheap, cascading folds are parallelizable; the resulting VIR is simpler and faster to optimize. The net goal is LLVM-class codegen quality with materially lower compile time.

1.1 Goals

  1. Codegen quality is the headline. Generated code quality takes priority over compile speed.
  2. Two-stage optimization. HIR performs small folds and trait derivation; VIR performs large transformations.
  3. Trait-driven VIR. VIR never re-analyzes; every fact it uses is an attribute, relationship, or request supplied by HIR.
  4. Parallel, cascading HIR. HIR passes run per-function in parallel and cascade to a fixpoint.
  5. Compiler-friendly C. The C backend always emits C written to be pattern-matched by GCC/Clang.
  6. Deterministic output. Same input + same config ⇒ byte-identical output, regardless of thread count.

1.2 Non-goals (v1)

  • Linking. libVCT produces object files; linking is the caller's job. The sole exception is -mangled (cosmopolitan) mode.
  • Windows / MSVC. Linux/POSIX-first; gcc and clang only.
  • LLVM bitcode. The LLVM backend emits textual .ll only; the caller runs llvm-as.
  • LLVM debug metadata. No !llvm.dbg in v1; debug mapping exists on the C path via #line.
  • A hand-authored VIR. VIR is internal and intentionally hostile to hand-authoring. A textual form exists solely for tools and round-trip tests (§9.8); it is not a supported authoring surface.
  • Consuming LLVM IR or Vox's existing IR. libVCT owns its IR definition.

1.3 Glossary

Term Meaning
HIR High-level IR: the frontend AST, annotated in place with traits.
VIR Vox IR: a low-level SSA/CFG instruction IR.
Trait The union of Attribute, Request, and Relationship attached to a node.
Attribute A fact about a node (type, value, mutability, ...).
Request An attribute-derived suggestion from HIR to VIR.
Relationship A directed edge between nodes, traceable up/down.
Cascade The chain of HIR rewrites triggered by one fold.
Lowering The "middleman" that translates annotated HIR into VIR SSA.
Remark An optimizer decision record emitted for diagnostics.

2. Component Architecture

2.1 Modules

Module Responsibility Allocation Visibility
vct.ir.hir HIR AST node types, Trait model, builder/emitter API arena public
vct.ir.vir VIR SSA/CFG types, VIR type system, instruction set arena internal
vct.traits Attribute / Request / Relationship definitions + queries arena public
vct.comptime Comptime evaluator / constant-folding engine arena internal
vct.hir.opt HIR pass framework + passes, parallel cascade arena internal
vct.lower HIR → VIR lowering + SSA construction arena internal
vct.vir.opt VIR pass framework + heavy passes arena internal
vct.backend.c VIR → C17 emitter (out-of-SSA, compiler-friendly C) arena + GC public entry
vct.backend.llvm VIR → textual LLVM IR emitter arena + GC public entry
vct.driver cc invocation, flag assembly, .o emission, LTO, cosmopolitan GC public
vct.cli Thin command-line wrapper GC public
vct.diag Diagnostics, source maps, error reporting GC public
vct.context Context / arena ownership, Config GC + arena public
vct.test.hirbuild Fluent HIR test-builder harness arena public

2.2 Data flow

frontend --build--> HIR
    --hir.opt (parallel, cascading)--> annotated HIR
    --lower--> VIR (SSA/CFG)
    --vir.opt--> optimized VIR
    --backend.c--> C17 source --driver--> .o          (default path)

optimized VIR --backend.llvm--> textual LLVM IR      (caller links LLVM)

2.3 Public interfaces

  1. HIR builder (frontend-facing). Construct nodes, set types/values, attach frontend-supplied traits (static, hints), and finish a module.
  2. Trait query (VIR-facing). Read attributes and walk relationships. This is the only channel through which VIR obtains facts.
  3. Backend (driver-facing). Consume optimized VIR and emit C (or textual LLVM IR).

2.4 Load-bearing boundaries

  • vct.traits is the contract. Both HIR and VIR depend on it. VIR depends on nothing else from HIR except node identity.
  • vct.lower is the SSA construction site. Phi-nodes, dominance, and the memory model all land here; it is the most algorithmically dense module.

3. The Trait Model

A Trait is the union of three sub-structures hung on HIR nodes, and the entire vocabulary VIR may reason with:

struct Trait {
    Attribute  attr;
    Request[]  reqs;
    Relation[] rels;
}

Traits are exposed publicly. A frontend MAY suggest traits to HIR; HIR validates suggestions and fills in correct values. Derived traits are authoritative and VIR trusts them blindly.

3.1 Attributes

Attributes are facts about a node, with two origins in one namespace:

  • Suggested — set by the frontend at build time. HIR validates; on mismatch it emits a warning (promoted to error under -Werror) and overwrites with the correct value.
  • Derived — produced by the HIR optimizer during the cascade. Authoritative.

Every attribute carries an attr_source bit recording whether it was suggested or derived.

v1 attribute set:

Attribute Meaning
ty The node's VIR type.
const_value Present iff the value is known at compile time.
is_static Frontend-declared compile-time value.
is_comptime HIR-proven compile-time value.
is_constant Value is constant (not necessarily comptime).
is_used Node has at least one use.
is_mutably_used Node is mutated through at least one use.
is_addressed Address taken by pointer/reference.
escapes Value escapes its defining scope.
is_runtime_mutable May change at runtime.
may_change_at_runtime The observable value may differ between reads/executions (external state). Distinct from is_runtime_mutable, which means the storage is written at runtime.
complex Frontend-provided; gates the NoOptimize request.

3.2 Requests

Requests are attribute-derived suggestions from HIR to VIR — never ad-hoc. Each has a strength:

  • Soft — a suggestion VIR MAY decline.
  • Strong — violating it is likely a bug (e.g., NoInline, MustTail, AlwaysInline, NoOptimize).

Every request produces a RequestResult:

struct RequestResult {
    Request    req;
    bool       accepted;
    DenyReason reason;
    string     note;   // one-line human-readable explanation
}

A denial MUST carry a DenyReason enum (for tooling) and a one-line note.

enum DenyReason {
    CostModel,        // profitable only under a different cost model
    Illegality,       // transform would be incorrect
    AlreadyDone,      // no-op: the goal already holds
    Unsupported,      // VIR does not implement it yet
    ContradictsTrait, // conflicts with a stronger fact
    TooLarge,         // exceeds a size budget
    NoTargetSupport,  // backend/target cannot express it
}

3.2.1 Request catalog (v1)

Inlining / call edges: Inline; AlwaysInline (Strong); NoInline (Strong); TailCall; MustTail (Strong); NoTail (Strong); Devirtualize; ColdCall; LikelyCall; UnlikelyCall.

Loops: Vectorize; Unroll(factor); Interleave(count); Peel(count); Distribute; Fuse; Jam; Unswitch; Rotate; LICM (action); LoopInvariant (property); MustProgress.

Memory effects: NoAlias; Restrict; NonNull; Align(n); ReadOnly; WriteOnly; NoRead; NoWrite; NoCapture; Dereferenceable(n); Constant; NoUndef (reserved); Prefetch.

Control flow: HotPath; ColdPath; Unreachable; NoReturn.

Assumptions: Assume(pred); Range(lo, hi).

Optimization control: NoOptimize (Strong; requested only when the frontend sets complex on the node).

3.2.2 Request conflict rules

  • Strong beats Soft.
  • Two conflicting Strong requests (e.g., Inline vs NoInline, TailCall vs NoTail) ⇒ HIR emits a diagnostic and rejects the frontend suggestion.
  • LoopInvariant (a proven property VIR may exploit) and LICM (an explicit request to hoist) are distinct and both retained.
  • NoRead/NoWrite are primitives on an access edge; ReadOnly/WriteOnly are region-level facts derived from them. All four are retained, with the scope difference documented.

3.3 Relationships

Relationships are directed edges HIR leaves for the non-constant world. They are traceable up and down to a terminal node.

Flags:

Flag Meaning
is_offspring Cannot trace up.
is_ancestor Cannot trace down.
is_common Branch point; VIR must choose a direction.
has_siblings Connects down to 2+ nodes.
is_apex Part of a cycle (a → b → c → d → a), usually from polymorphism.

Relation attributes (small set): was_changed; is_mutable; is_pointer.

An apex SHOULD normally be ironed out by HIR; residual apexes typically arise from polymorphism.

3.4 Representation & staleness

  • Representation (resolved): an embedded dense Trait struct on every node is primary, plus an optional sparse side-channel for rare / frontend-extensible attributes.
  • Staleness (resolved): epoch/generation counters with dirty propagation along relationships. A trait is valid only for the node's current epoch. This survives parallel HIR passes.

4. HIR Optimizer

Role: discovery + small folds + trait derivation. HIR hands VIR a "dictionary".

4.1 Pass framework

interface HirPass {
    string name();
    void run(HirFunction fn, HirContext ctx);
}

Passes mutate the annotated AST in place. Successful rewrites emit triggers:

enum Trigger {
    ConstantUnfolded,
    UsesReplaced,
    NodeDeleted,
    TraitChanged,
    StaticDiscovered,
}

A worklist scheduler collects (pass, node) pairs, dedupes, and runs to a fixpoint. A per-function step budget guarantees termination.

4.2 The cascade

Canonical chain:

4 * 16
  -> const 64
  -> mark is_comptime / is_constant / const_value = 64
  -> prove not addressed / not runtime-mutable
  -> replace all uses with literal 64
  -> node dead -> delete
  -> downstream nodes now constant -> enqueue

Each hop emits a trigger; the worklist drains until nothing new is provable. HIR never inlines — it only proves smallness and issues an Inline request to VIR.

4.3 Parallelism & determinism

  • Parallelize per function (independent HIR trees).
  • Within a function, passes run sequentially so cascade order is deterministic.
  • Cross-function effects (inlining, global constant propagation) use a module-level fixpoint: parallel per-function passes → deterministic module phase → re-enqueue only affected functions.
  • Epoch counters prevent stale trait reads in parallel workers.

4.4 Comptime evaluator

An interpreter over HIR subgraphs. Eligible only if: no side effects, primitives only, no global mutation, no I/O, and within loop/recursion budgets. Otherwise it bails and defers to VIR. Outputs const_value, then is_comptime + is_constant.

The frontend static attribute seeds the evaluator. HIR trusts static as "this is compile-time" but still validates it.

Budgets (configurable; defaults):

Budget Default
comptime.max_iterations 4096
comptime.max_recursion_depth 256
comptime.max_steps 1,000,000
comptime.max_aggregate_elements 65,536
hir.max_rewrites_per_function 100,000
hir.max_rewrites_per_module 1,000,000

All are overridable via the API and CLI.

4.5 Defer

HIR carries a real high-level defer statement. The optimizer inlines the deferred call at every scope-exit path and then erases the defer marker — zero runtime tax.

4.6 Optimization-level behavior

Level HIR VIR
-O0 traits validated only off
-O1 builtin folds + discovery off
-O2 full cascade + comptime on
-O3 + special AST transforms (loop-unfold/vectorize → constant-unfold → comptime) on
-Ofast = -O3 + -march=native + fast-math on
-Oz size-tuned on

Compiler-friendly C is emitted at ALL optimization levels. -O3 merely adds more aggressive AST transforms.

5. Lowering (HIR → VIR)

Role: the "middleman". Pure translation + SSA construction; no discovery.

5.1 Contract

  • Input: annotated HIR with complete, epoch-valid traits.
  • Output: VIR — basic blocks, phi-nodes, def-use, SSA.
  • Lowering validates trait completeness. A missing or contradictory trait raises an internal-compiler-error diagnostic (see §10.5).
  • Deterministic; parallelizable per function.

5.2 Form

Lowering is a streaming recursive interpreter over the HIR tree ("tcc-for-VIR"):

  • Structured HIR (if/while/for/switch/block) → blocks + terminators (br/condbr/switch/ret/unreachable).
  • Expressions → temporaries + instructions.

5.3 SSA construction

SSA is built with the Braun et al. sealed-block algorithm: one pass, on-the-fly phi insertion, no separate dominance-frontier pass. A loop header is sealed once its back-edge is emitted.

5.4 Memory model (trait-driven hybrid)

Condition Representation
scalar ∧ ¬is_addressed ∧ ¬escapes ∧ ¬is_runtime_mutable pure SSA value
addressed / escaping / runtime-mutable / aggregate alloca + explicit load/store
volatile / atomic forced memory, never promoted

VIR's mem2reg/SROA MAY promote memory back to SSA when traits confirm safety.

5.5 Phi

VIR has an explicit LLVM-style Phi instruction whose operands are (value, predecessor-block) pairs.

5.6 Identity & traits

A LoweringMap (HIR node → VIR entity(ies)) is built during lowering. Each VIR entity carries its own Trait, populated from the source HIR node. HIR Relationship endpoints are rewritten to VIR entities via the map. VIR depends only on vct.traits + node identity.

5.7 Optimization barriers

complex/NoOptimize nodes are lowered inside a region-level optimization barrier. VIR passes MUST NOT rewrite across it.

5.8 Defer fallback

If HIR optimization is off (-O0/-O1), lowering itself expands defer at every scope exit.

5.9 VIR type system

More minimal than HIR but still fleshed out: integers with width, floats, pointers, aggregates, function types, void.

5.10 Output invariants

Well-formed SSA:

  • every use is dominated by its def,
  • every block is terminated,
  • phi arity equals predecessor count,
  • a single def per value.

Verified in debug builds (§11).

6. VIR Optimizer

Role: the heavy-lifting half. Reads traits only, transforms SSA, performs no discovery. Runs at -O2/-O3/-Ofast/-Oz.

6.1 Contract

  • Input: lowered VIR (well-formed SSA/CFG) + complete traits.
  • Output: optimized VIR, still in SSA.
  • Strict trait consumer: a transform is enabled by an attribute/relationship or requested by a Request. VIR MUST NOT assume a fact absent from traits.

6.2 Framework

interface VirPass {
    string name();
    void run(VirModule m, PassContext ctx);
}

Analyses: DominatorTree, PostDominatorTree, LoopInfo, AliasInfo, DefUse, CallGraph, RangeInfo. Passes declare which analyses they preserve; the manager invalidates the rest (fine-grained, not invalidate-all). Epoch counters catch stale trait reads.

6.3 Default pipeline (-O2)

A module pass manager interleaving function passes and IPA:

  1. Canonicalize: mem2reg, SROA, instcombine, simplifyCFG, early-CSE, DCE.
  2. Scalar: GVN, SCCP, LICM, indvars, reassociation.
  3. IPA: inliner (driven by Inline/AlwaysInline/NoInline/ColdCall/LikelyCall), global DCE, IPSCCP, function-attribute propagation.
  4. Loops: unroll/interleave/peel/rotate/unswitch/distribute/fuse/jam, each gated by its Request.
  5. Vectorize: loop + SLP, gated by Vectorize/LoopInvariant/Range/NoAlias/Restrict.
  6. Memory: alias-driven DSE, GEP simplification, load widening.
  7. Control flow: block layout (HotPath/ColdPath), tail-call formation (TailCall/MustTail/NoTail), unreachable pruning, jump threading.
  8. Codegen prep: canonicalize for the backend; remains SSA.

-O3/-Ofast raise aggression. -Oz is size-first (most unrolling/vectorization off).

6.4 Requests

Each Request yields a RequestResult (§3.2). Denials carry a DenyReason + one-line note.

6.5 Alias analysis

Synthesized from NoAlias/Restrict/NoCapture/Dereferenceable/ReadOnly/WriteOnly + provenance — no guessing.

6.6 Barriers

Region optimization barriers (complex/NoOptimize, §5.7) are opaque; passes skip across them.

6.7 Budgets

vir.max_iterations, vir.max_pipeline_rounds (configurable).

6.8 Resolutions

  • A Strong-request denial is a hard error when the backend is capable and the reason is Illegality/ContradictsTrait; it is a warning for Unsupported/NoTargetSupport.
  • The pipeline is a fixed canonical pipeline (LLVM-style), not adaptive request-driven ordering.
  • Inlining is a VIR/IPA transform (HIR only requests it).
  • RangeInfo is lightweight, seeded by Range/Assume traits — no full ScalarEvolution.
  • Analysis invalidation is fine-grained (preserved-analyses sets).

7. C Backend (vct.backend.c)

Role: VIR → C17.

7.1 Contract

  • Input: optimized VIR (still SSA, traits attached).
  • Output: one C17 translation unit per function.
  • Emission is per-function independent → parallel; the driver concatenates deterministically.
  • Two-pass emission: forward declarations, then definitions.

7.2 Out-of-SSA

Boissinot et al., "Revisiting Out-of-SSA Translation" — handles the lost-copy and swap problems. Critical edges are split before out-of-SSA.

7.3 Compiler-friendly C

  • Structured control-flow reconstruction (if/else/while/for/do/switch) for reducible CFGs; goto fallback for irreducible.
  • Qualifiers from traits: restrict (Restrict/NoAlias), const (ReadOnly/NoWrite), _Noreturn (NoReturn), cold/hot (ColdPath/HotPath), static inline / noinline.
  • Hints: __builtin_expect; __builtin_assume_aligned / _Alignas (Align); __builtin_unreachable (Unreachable); __builtin_assume (clang, Assume); #pragma GCC ivdep / #pragma clang loop vectorize(enable) (Vectorize); pure/const (NoRead/NoWrite).
  • NoOptimize/complex barrier: a compiler fence — __asm__ __volatile__("" ::: "memory") plus an opaque noinline call.

7.4 Types & layout

intN_t/uintN_t, float/double, T*, struct/union/arrays, function pointers, void. VIR aggregate GEPs use field indices, not byte offsets; the C compiler picks layout. An explicit layout trait pins the ABI (emits padding + _Static_assert).

7.5 Naming

Deterministic mangling. Exported names preserved; internal names sanitized and uniquified. A reserved-word/collision prefix table.

7.6 Intrinsics

Mostly __builtin_* (GCC/Clang), plus a small set of portable fallback helpers for what builtins do not cover.

7.7 Debug mapping

#line directives back to frontend source, flag-gated (on for debug builds, off for release). The frontend registers its source map with vct.diag.

7.8 ABI

Exported functions match the C ABI; internal functions are static; struct-by-value follows the C compiler's ABI. MustTail requires [[clang::musttail]] on Clang; on GCC a MustTail request is a hard error.

8. LLVM Backend (vct.backend.llvm)

Role: VIR → textual LLVM IR, post VIR optimization. Pure translation; no optimization or discovery. Output is a self-contained .ll module. libVCT never invokes llvm-as/opt/llc or links LLVM — that is the frontend's job. Deterministic per-function emission with stable concatenation.

8.1 SSA mapping

VIR is already SSA → 1:1 mapping: instruction → instruction, block → block, Phi → phi, terminators, alloca/load/store. No out-of-SSA (the opposite of the C backend).

8.2 Types

iN integers, float/double, opaque ptr (not typed), struct/array (VIR field indices → LLVM struct indices), void, function types.

8.3 Traits → LLVM attributes/metadata

Trait LLVM
Restrict/NoAlias noalias + !alias.scope/!noalias
ReadOnly/WriteOnly/NoRead/NoWrite readonly/writeonly
NonNull nonnull
Align(n) align n
Range !range on loads
Assume llvm.assume
NoInline/AlwaysInline noinline/alwaysinline
TailCall/MustTail/NoTail tail/musttail/none
NoReturn noreturn
HotPath/ColdPath !prof
Vectorize !llvm.loop.vectorize.enable
Unroll(n) !llvm.loop.unroll.count
Unreachable unreachable
NoUndef reserved; only meaningful here

8.4 Intrinsics

memcpy/memset/memmove, llvm.sqrt.*, llvm.fabs.*, llvm.ctpop.*, llvm.fshl.*, saturating/overflow ops.

8.5 Module scaffolding

Target triple (from -target) + derived target datalayout, so the module is self-contained. Declarations precede definitions.

8.6 Resolutions

  • Pin LLVM 18+ only (opaque pointers, current metadata).
  • No !llvm.dbg in v1; debug only on the C path via #line.
  • Emit text .ll only; the caller runs llvm-as.
  • Emit real llvm.assume / !range.

9. Driver & CLI

9.1 Role split

  • vct.driver — library-level orchestrator: takes emitted C TUs, invokes the C compiler, collects .o. Owns the -llvm path (returns LLVM IR, never touches cc) and the -mangled path.
  • vct.cli — thin wrapper: parses flags into Config, drives the whole pipeline (read IR → HIR opt → lower → VIR opt → backend.c → driver), formats diagnostics, sets exit codes. GC-allocated; holds no optimizer state.

9.2 Pipeline

read input
  -> (HIR opt if enabled)
  -> lower
  -> (VIR opt if enabled)
  -> backend.c -> driver -> .o

-S stops after C; -emit-llvm/-llvm stop after LLVM IR.

9.3 Flag surface (v1)

Group Flags
Optimize -O0 -O1 -O2 -O3 -Ofast -Oz
Target -march=<arch> (→ cflags), -mcpu=<cpu>, -target <triple>
Modes -S (C only), -emit-llvm/-llvm, -mangled (cosmopolitan)
Toolchain -cc=<gcc|clang> (default auto), -cflags="...", -j<N>, -save-temps
LTO -flto[=full|thin], -ffat-lto-objects
Debug -g, --dump-hir, --dump-vir, --verify, --time-passes, --stats
Output -o <path>
Warnings -Wall -Werror -Wextra always on; -Wno-error escape hatch; -w to silence
Diagnostics --remarks, --diagnostics=json
Pass toggles every pass exposed as -f<pass> / -fno-<pass>

9.4 cc invocation

cc -std=c17 -c <tu.c> -o <tu.o> <cflags> <march> -Wall -Werror -Wextra <g> <flto>

One process per TU, bounded by -j. The compiler is discovered via $CC then PATH; its version/dialect is probed once and drives the backend's pragma/builtin choices (§7.3, §7.6).

9.5 Error handling

cc stderr is captured. #line directives (§7.7) map C errors back to VIR/source; they are re-emitted via vct.diag. A cc failure produces a diagnostic and a nonzero exit.

9.6 Determinism

Stable TU ordering and flag ordering. Temp files live under a GC-managed temp dir (cleaned unless -save-temps).

9.7 -mangled

Switches the toolchain to cosmocc/cosmopolitan libc and produces an Actually Portable Executable. This mode does link — the opt-in exception to "caller links".

9.8 Resolutions

  • Driver compiles to .o only; a single -o merges per-function objects with an ld -r relocatable link; a recommended link line / response file is emitted for the caller. -mangled is the sole mode producing a final executable.
  • The CLI also reads a textual HIR/VIR format for testing/tools/round-trips; frontends still embed the library.
  • Every pass is exposed as a -f/-fno toggle.
  • -Wall -Werror -Wextra default; -Wno-error escape hatch; -Werror applies to libVCT-generated C.

10. Diagnostics & Source Maps (vct.diag)

10.1 Central service

One central diagnostics service; every module emits through it, nothing prints directly. GC-allocated.

struct Diagnostic {
    Severity    severity;   // Error | Warning | Note | Remark | Ice
    DiagCode    code;       // stable namespaced, e.g. VCT1002
    string      message;
    Span        primary;
    Span[]      notes;
    Suggestion[] suggestions;
}

10.2 Source maps

The frontend registers SourceLocation{file,line,col,len} against HIR nodes; VIR resolves via the LoweringMap (§5.6). Degradation to no-location is graceful. The C backend emits #line (§7.7); cc errors are parsed and re-mapped (§9.5).

10.3 Channels

  • DiagnosticConsumer callback (canonical for embedded use).
  • Human-readable CLI (color when tty).
  • JSON (--diagnostics=json, versioned schema, shipped in v1).

10.4 Optimizer remarks

--remarks (off by default; automatically enabled at -O3) emits per-decision remarks tied to the trait pipeline, e.g.:

  • HIR: replaced 3 uses of node#412 (comptime 64)
  • VIR: denied Inline on @foo — CostModel (callee 2.4x size budget)

10.5 ICE policy

Internal compiler errors include: missing/contradictory trait at lowering (§5.1), VIR invariant violated in debug (§5.10), and a Strong request denied for Illegality/ContradictsTrait (§6.8).

  • Release: return a failure result + ICE diagnostic with a module-dump request.
  • Debug: assert.

Never silently continue.

10.6 Exit codes

Code Meaning
0 success
1 diagnostics present
2 usage/config error
3 ICE

10.7 Resolutions

  • The trait-mismatch warning is promoted to error under -Werror (frontends can opt out with -Wno-error=).
  • ICE in release = diagnostic + error result.
  • Remarks: both opt-in --remarks and automatically enabled at -O3.
  • Ship a versioned JSON diagnostics format in v1.

11. Testing & Verification

11.1 Core constraint

VIR is a strict trait consumer and cannot be tested without a valid HIR fixture. The test strategy is anchored on a fixture layer.

11.2 Layers

  1. Unit tests — D unittest per module.
  2. HIR test-builder harness (vct.test.hirbuild) — a fluent builder constructing HIR modules + traits without a frontend. This is the fixture layer for HIR-opt, lowering, VIR-opt, and both backends.
  3. Textual round-trip — property: print(parse(print(m))) == print(m).
  4. FileCheck-style tests — CHECK / CHECK-NOT / CHECK-NEXT directives.
  5. Verifier — structural SSA (§5.10) + trait consistency, via --verify; asserted after every pass in debug.
  6. Differential — the VIR reference interpreter vs the compiled .o; the C path vs the LLVM path.
  7. Fuzz — random HIR/trait combinations; property: terminates, verifier clean, output compiles.
  8. Matrix — gcc + clang × -O0..-Ofast/-Oz × C/LLVM.
  9. Perf benchmarks — codegen quality is the headline; compile time tracked via --time-passes/--stats.

11.3 VIR reference interpreter

Approved as v1 scope. It is the differential oracle: the same VIR is run through the interpreter and through the compiled .o, and results are compared.

11.4 Determinism

Every pipeline run is executed twice in CI; output must be byte-identical.

11.5 Regression corpus

A corpus of input IR + expected FileCheck patterns; CI runs the full matrix.

11.6 Resolutions

  • FileCheck-style matcher (not exact golden files).
  • Build the VIR reference interpreter in v1 as a differential oracle.
  • Verifier always-on in debug builds; --verify opt-in in release.
  • Random-HIR + random-trait fuzzers in v1.

12. Build, Packaging & Public API Lifecycle

12.1 Language & toolchain

Written in D. LDC2 is used for release builds (best codegen); DMD is used for debug builds (faster compile/iteration). Zero runtime dependencies — only a D compiler to build, and a system C compiler discovered at runtime for the C path.

12.2 Build system

xmake is the build system — not dub. A single xmake.lua drives all targets: the release profile uses LDC2, the debug profile uses DMD. No external package manager is required to build.

12.3 Artifacts

  • libvct.a (static) and libvct.so (shared).
  • vct CLI binary.
  • vctc.h — C API header.
  • vct.test.hirbuild — shipped publicly.

12.4 API surfaces

  • Thin extern(C) C API (v1, the primary public surface): opaque handles for Context/Module/Builder/Config, functions to build IR, run the pipeline, and query diagnostics. No logic lives in the shim.
  • Native D API (full feature set): the implementation surface; the C API is a thin shim over it, so the D API is exercised by every C API call.

12.5 Module visibility

  • Public: vct.ir.hir, vct.traits, vct.context, vct.backend.c entry, vct.backend.llvm entry, vct.driver, vct.diag, vct.test.hirbuild.
  • Internal: vct.ir.vir, vct.lower, vct.hir.opt, vct.vir.opt, vct.comptime internals.

12.6 Memory ownership & lifecycle

  • A Context owns one or more arenas; an arena is the unit of reclamation. Default: one arena per Module, freed wholesale.
  • All IR and optimizer objects are arena-owned and non-GC; passes mutate in place under epoch guards.
  • GC is permitted only on cold paths (driver, diagnostics, CLI).
  • Arena hooks are exposed: Context accepts an ArenaAllocator interface (allocate/reset/destroy) so embedders can supply their own backing memory; a default bump allocator ships in-tree.

12.7 Config

One struct holds opt level, target (-march/-mcpu/-target), all budgets, pass toggles, warning policy, cc selection, and mode (-S/-llvm/-mangled). The CLI is a pure function from argv to Config; the library takes a Config. Programmatic callers bypass the CLI.

12.8 Threading contract

Thread-safe when each compilation unit has its own Context/arenas; no shared mutable global state. Intra-module parallelism is internal and bounded by -j.

12.9 Versioning

  • Library semver.
  • Textual IR format version — separate and independently versioned.
  • Trait vocabulary version — adding attributes/requests is backward-compatible; changing semantics is a major bump. Experimental passes are gated behind feature flags.

12.10 Determinism guarantee

Same Config + same input ⇒ byte-identical output, independent of thread count.

12.11 Resolutions

  • Thin C API first (the primary public surface in v1).
  • ArenaAllocator hooks exposed.
  • LDC2 for release builds; DMD for debug builds.
  • xmake as the build system (not dub).

13. Worked End-to-End Example

This example is normative for trait/request semantics.

Input program:

fn foo(x: int, y: int) -> int {
    return x + y;
}

fn main() -> int {
    x = 4;
    y = x + 4;
    z = foo(x, y);
    a = sqrt(z);
    println(a);
}

The HIR cascade proceeds as follows:

  1. HIR comptime-evaluates x = 4 → is_comptime, is_constant, const_value = 4.
  2. y = x + 4 unfolds to y = 8 → is_comptime.
  3. foo(x, y) is proven small → HIR issues an Inline request; after inlining, z = 4 + 8 = 12 → comptime.
  4. a = sqrt(z) → a = sqrt(12) comptime → a = 3.4641016151377544.
  5. The whole program collapses to println(3.4641016151377544).

VIR then optimizes the println call. The emitted C is effectively a single call with the folded constant. This demonstrates trait propagation HIR → VIR and the request lifecycle (Inline accepted).

14. Open Questions & Future Work

  • Debug info on the LLVM path (!llvm.dbg) — deferred past v1.
  • Windows / MSVC support — out of scope for v1.
  • LLVM bitcode emission — deferred; caller runs llvm-as.
  • Incremental / cached compilation — content-hash .o caching is out of scope for v1.
  • Cross-compilation targets beyond -target passthrough and cosmopolitan — future work.
Reference in New Issue View Git Blame Copy Permalink