docs: add libVCT architecture specification
This commit is contained in:
@@ -0,0 +1,788 @@
|
||||
# libVCT — Vox C Transpiler Library: Architecture Specification
|
||||
|
||||
- **Status:** Draft (pending final review)
|
||||
- **Date:** 2026-09-13
|
||||
- **Scope:** Whole-library architecture — component boundaries and interfaces. Subsystem internals are deferred to later specs.
|
||||
- **Normative language:** `MUST`, `SHOULD`, `MAY` per RFC 2119.
|
||||
|
||||
---
|
||||
|
||||
## 1. Overview
|
||||
|
||||
libVCT is an LLVM-inspired compiler library written in D. It consumes an IR, optimizes it, and transpiles it into C17, then drives a system C compiler to produce an object file. An alternate backend emits textual LLVM IR instead of C.
|
||||
|
||||
The library is organized around a **two-stage IR**:
|
||||
|
||||
- **HIR** — a high-level, tree-shaped AST that is the frontend's output and is **annotated in place** with *traits*. HIR does discovery and small, cheap folds.
|
||||
- **VIR** — a low-level SSA/CFG instruction IR. VIR does the heavy lifting and performs **no discovery**; it is a strict consumer of HIR-provided traits.
|
||||
|
||||
The design thesis: because HIR hands VIR a fully annotated "dictionary" of facts, VIR can skip analysis and go straight to transformation. HIR's cheap, cascading folds are parallelizable; the resulting VIR is simpler and faster to optimize. The net goal is LLVM-class codegen quality with materially lower compile time.
|
||||
|
||||
### 1.1 Goals
|
||||
|
||||
1. **Codegen quality is the headline.** Generated code quality takes priority over compile speed.
|
||||
2. **Two-stage optimization.** HIR performs small folds and trait derivation; VIR performs large transformations.
|
||||
3. **Trait-driven VIR.** VIR never re-analyzes; every fact it uses is an attribute, relationship, or request supplied by HIR.
|
||||
4. **Parallel, cascading HIR.** HIR passes run per-function in parallel and cascade to a fixpoint.
|
||||
5. **Compiler-friendly C.** The C backend always emits C written to be pattern-matched by GCC/Clang.
|
||||
6. **Deterministic output.** Same input + same config ⇒ byte-identical output, regardless of thread count.
|
||||
|
||||
### 1.2 Non-goals (v1)
|
||||
|
||||
- **Linking.** libVCT produces object files; linking is the caller's job. The sole exception is `-mangled` (cosmopolitan) mode.
|
||||
- **Windows / MSVC.** Linux/POSIX-first; gcc and clang only.
|
||||
- **LLVM bitcode.** The LLVM backend emits textual `.ll` only; the caller runs `llvm-as`.
|
||||
- **LLVM debug metadata.** No `!llvm.dbg` in v1; debug mapping exists on the C path via `#line`.
|
||||
- **A user-facing VIR.** VIR is internal and intentionally hostile to hand-authoring.
|
||||
- **Consuming LLVM IR or Vox's existing IR.** libVCT owns its IR definition.
|
||||
|
||||
### 1.3 Glossary
|
||||
|
||||
| Term | Meaning |
|
||||
|---|---|
|
||||
| **HIR** | High-level IR: the frontend AST, annotated in place with traits. |
|
||||
| **VIR** | Vox IR: a low-level SSA/CFG instruction IR. |
|
||||
| **Trait** | The union of Attribute, Request, and Relationship attached to a node. |
|
||||
| **Attribute** | A fact about a node (type, value, mutability, ...). |
|
||||
| **Request** | An attribute-derived suggestion from HIR to VIR. |
|
||||
| **Relationship** | A directed edge between nodes, traceable up/down. |
|
||||
| **Cascade** | The chain of HIR rewrites triggered by one fold. |
|
||||
| **Lowering** | The "middleman" that translates annotated HIR into VIR SSA. |
|
||||
| **Remark** | An optimizer decision record emitted for diagnostics. |
|
||||
|
||||
---
|
||||
|
||||
## 2. Component Architecture
|
||||
|
||||
### 2.1 Modules
|
||||
|
||||
| Module | Responsibility | Allocation | Visibility |
|
||||
|---|---|---|---|
|
||||
| `vct.ir.hir` | HIR AST node types, Trait model, builder/emitter API | arena | **public** |
|
||||
| `vct.ir.vir` | VIR SSA/CFG types, VIR type system, instruction set | arena | internal |
|
||||
| `vct.traits` | Attribute / Request / Relationship definitions + queries | arena | **public** |
|
||||
| `vct.comptime` | Comptime evaluator / constant-folding engine | arena | internal |
|
||||
| `vct.hir.opt` | HIR pass framework + passes, parallel cascade | arena | internal |
|
||||
| `vct.lower` | HIR → VIR lowering + SSA construction | arena | internal |
|
||||
| `vct.vir.opt` | VIR pass framework + heavy passes | arena | internal |
|
||||
| `vct.backend.c` | VIR → C17 emitter (out-of-SSA, compiler-friendly C) | arena + GC | **public entry** |
|
||||
| `vct.backend.llvm` | VIR → textual LLVM IR emitter | arena + GC | **public entry** |
|
||||
| `vct.driver` | cc invocation, flag assembly, `.o` emission, LTO, cosmopolitan | GC | **public** |
|
||||
| `vct.cli` | Thin command-line wrapper | GC | **public** |
|
||||
| `vct.diag` | Diagnostics, source maps, error reporting | GC | **public** |
|
||||
| `vct.context` | Context / arena ownership, `Config` | GC + arena | **public** |
|
||||
| `vct.test.hirbuild` | Fluent HIR test-builder harness | arena | **public** |
|
||||
|
||||
### 2.2 Data flow
|
||||
|
||||
```
|
||||
frontend --build--> HIR
|
||||
--hir.opt (parallel, cascading)--> annotated HIR
|
||||
--lower--> VIR (SSA/CFG)
|
||||
--vir.opt--> optimized VIR
|
||||
--backend.c--> C17 source --driver--> .o (default path)
|
||||
|
||||
optimized VIR --backend.llvm--> textual LLVM IR (caller links LLVM)
|
||||
```
|
||||
|
||||
### 2.3 Public interfaces
|
||||
|
||||
1. **HIR builder (frontend-facing).** Construct nodes, set types/values, attach frontend-supplied traits (`static`, hints), and finish a module.
|
||||
2. **Trait query (VIR-facing).** Read attributes and walk relationships. This is the **only** channel through which VIR obtains facts.
|
||||
3. **Backend (driver-facing).** Consume optimized VIR and emit C (or textual LLVM IR).
|
||||
|
||||
### 2.4 Load-bearing boundaries
|
||||
|
||||
- **`vct.traits` is the contract.** Both HIR and VIR depend on it. VIR depends on nothing else from HIR except **node identity**.
|
||||
- **`vct.lower` is the SSA construction site.** Phi-nodes, dominance, and the memory model all land here; it is the most algorithmically dense module.
|
||||
|
||||
---
|
||||
|
||||
## 3. The Trait Model
|
||||
|
||||
A `Trait` is the union of three sub-structures hung on HIR nodes, and the entire vocabulary VIR may reason with:
|
||||
|
||||
```d
|
||||
struct Trait {
|
||||
Attribute attr;
|
||||
Request[] reqs;
|
||||
Relation[] rels;
|
||||
}
|
||||
```
|
||||
|
||||
Traits are **exposed publicly**. A frontend MAY **suggest** traits to HIR; HIR validates suggestions and fills in correct values. Derived traits are authoritative and VIR trusts them blindly.
|
||||
|
||||
### 3.1 Attributes
|
||||
|
||||
Attributes are facts about a node, with **two origins in one namespace**:
|
||||
|
||||
- **Suggested** — set by the frontend at build time. HIR **validates**; on mismatch it emits a warning (promoted to error under `-Werror`) and **overwrites** with the correct value.
|
||||
- **Derived** — produced by the HIR optimizer during the cascade. Authoritative.
|
||||
|
||||
Every attribute carries an `attr_source` bit recording whether it was suggested or derived.
|
||||
|
||||
**v1 attribute set:**
|
||||
|
||||
| Attribute | Meaning |
|
||||
|---|---|
|
||||
| `ty` | The node's VIR type. |
|
||||
| `const_value` | Present iff the value is known at compile time. |
|
||||
| `is_static` | Frontend-declared compile-time value. |
|
||||
| `is_comptime` | HIR-proven compile-time value. |
|
||||
| `is_constant` | Value is constant (not necessarily comptime). |
|
||||
| `is_used` | Node has at least one use. |
|
||||
| `is_mutably_used` | Node is mutated through at least one use. |
|
||||
| `is_addressed` | Address taken by pointer/reference. |
|
||||
| `escapes` | Value escapes its defining scope. |
|
||||
| `is_runtime_mutable` | May change at runtime. |
|
||||
| `may_change_at_runtime` | Alias of intent: value is not frozen. |
|
||||
| `complex` | Frontend-provided; gates the `NoOptimize` request. |
|
||||
|
||||
### 3.2 Requests
|
||||
|
||||
Requests are **attribute-derived** suggestions from HIR to VIR — never ad-hoc. Each has a **strength**:
|
||||
|
||||
- `Soft` — a suggestion VIR MAY decline.
|
||||
- `Strong` — violating it is likely a bug (e.g., `NoInline`, `MustTail`, `AlwaysInline`, `NoOptimize`).
|
||||
|
||||
Every request produces a `RequestResult`:
|
||||
|
||||
```d
|
||||
struct RequestResult {
|
||||
Request req;
|
||||
bool accepted;
|
||||
DenyReason reason;
|
||||
string note; // one-line human-readable explanation
|
||||
}
|
||||
```
|
||||
|
||||
A denial MUST carry a `DenyReason` enum (for tooling) and a one-line `note`.
|
||||
|
||||
```d
|
||||
enum DenyReason {
|
||||
CostModel, // profitable only under a different cost model
|
||||
Illegality, // transform would be incorrect
|
||||
AlreadyDone, // no-op: the goal already holds
|
||||
Unsupported, // VIR does not implement it yet
|
||||
ContradictsTrait, // conflicts with a stronger fact
|
||||
TooLarge, // exceeds a size budget
|
||||
NoTargetSupport, // backend/target cannot express it
|
||||
}
|
||||
```
|
||||
|
||||
#### 3.2.1 Request catalog (v1)
|
||||
|
||||
**Inlining / call edges:** `Inline`; `AlwaysInline` (Strong); `NoInline` (Strong); `TailCall`; `MustTail` (Strong); `NoTail` (Strong); `Devirtualize`; `ColdCall`; `LikelyCall`; `UnlikelyCall`.
|
||||
|
||||
**Loops:** `Vectorize`; `Unroll(factor)`; `Interleave(count)`; `Peel(count)`; `Distribute`; `Fuse`; `Jam`; `Unswitch`; `Rotate`; `LICM` (action); `LoopInvariant` (property); `MustProgress`.
|
||||
|
||||
**Memory effects:** `NoAlias`; `Restrict`; `NonNull`; `Align(n)`; `ReadOnly`; `WriteOnly`; `NoRead`; `NoWrite`; `NoCapture`; `Dereferenceable(n)`; `Constant`; `NoUndef` (reserved); `Prefetch`.
|
||||
|
||||
**Control flow:** `HotPath`; `ColdPath`; `Unreachable`; `NoReturn`.
|
||||
|
||||
**Assumptions:** `Assume(pred)`; `Range(lo, hi)`.
|
||||
|
||||
**Optimization control:** `NoOptimize` (Strong; requested **only** when the frontend sets `complex` on the node).
|
||||
|
||||
#### 3.2.2 Request conflict rules
|
||||
|
||||
- `Strong` beats `Soft`.
|
||||
- Two conflicting `Strong` requests (e.g., `Inline` vs `NoInline`, `TailCall` vs `NoTail`) ⇒ HIR emits a diagnostic and **rejects** the frontend suggestion.
|
||||
- `LoopInvariant` (a proven property VIR may exploit) and `LICM` (an explicit request to hoist) are **distinct** and both retained.
|
||||
- `NoRead`/`NoWrite` are **primitives on an access edge**; `ReadOnly`/`WriteOnly` are **region-level facts** derived from them. All four are retained, with the scope difference documented.
|
||||
|
||||
### 3.3 Relationships
|
||||
|
||||
Relationships are directed edges HIR leaves for the non-constant world. They are traceable **up and down** to a terminal node.
|
||||
|
||||
**Flags:**
|
||||
|
||||
| Flag | Meaning |
|
||||
|---|---|
|
||||
| `is_offspring` | Cannot trace up. |
|
||||
| `is_ancestor` | Cannot trace down. |
|
||||
| `is_common` | Branch point; VIR must choose a direction. |
|
||||
| `has_siblings` | Connects down to 2+ nodes. |
|
||||
| `is_apex` | Part of a cycle (`a → b → c → d → a`), usually from polymorphism. |
|
||||
|
||||
**Relation attributes (small set):** `was_changed`; `is_mutable`; `is_pointer`.
|
||||
|
||||
An apex SHOULD normally be ironed out by HIR; residual apexes typically arise from polymorphism.
|
||||
|
||||
### 3.4 Representation & staleness
|
||||
|
||||
- **Representation (resolved):** an **embedded dense `Trait` struct** on every node is primary, plus an **optional sparse side-channel** for rare / frontend-extensible attributes.
|
||||
- **Staleness (resolved):** **epoch/generation counters** with dirty propagation along relationships. A trait is valid only for the node's current epoch. This survives parallel HIR passes.
|
||||
|
||||
---
|
||||
|
||||
## 4. HIR Optimizer
|
||||
|
||||
**Role:** discovery + small folds + trait derivation. HIR hands VIR a "dictionary".
|
||||
|
||||
### 4.1 Pass framework
|
||||
|
||||
```d
|
||||
interface HirPass {
|
||||
string name();
|
||||
void run(HirFunction fn, HirContext ctx);
|
||||
}
|
||||
```
|
||||
|
||||
Passes mutate the annotated AST in place. Successful rewrites emit **triggers**:
|
||||
|
||||
```d
|
||||
enum Trigger {
|
||||
ConstantUnfolded,
|
||||
UsesReplaced,
|
||||
NodeDeleted,
|
||||
TraitChanged,
|
||||
StaticDiscovered,
|
||||
}
|
||||
```
|
||||
|
||||
A **worklist scheduler** collects `(pass, node)` pairs, dedupes, and runs to a fixpoint. A per-function step budget guarantees termination.
|
||||
|
||||
### 4.2 The cascade
|
||||
|
||||
Canonical chain:
|
||||
|
||||
```
|
||||
4 * 16
|
||||
-> const 64
|
||||
-> mark is_comptime / is_constant / const_value = 64
|
||||
-> prove not addressed / not runtime-mutable
|
||||
-> replace all uses with literal 64
|
||||
-> node dead -> delete
|
||||
-> downstream nodes now constant -> enqueue
|
||||
```
|
||||
|
||||
Each hop emits a trigger; the worklist drains until nothing new is provable. **HIR never inlines** — it only proves smallness and issues an `Inline` request to VIR.
|
||||
|
||||
### 4.3 Parallelism & determinism
|
||||
|
||||
- Parallelize **per function** (independent HIR trees).
|
||||
- Within a function, passes run **sequentially** so cascade order is deterministic.
|
||||
- Cross-function effects (inlining, global constant propagation) use a **module-level fixpoint**: parallel per-function passes → deterministic module phase → re-enqueue only affected functions.
|
||||
- Epoch counters prevent stale trait reads in parallel workers.
|
||||
|
||||
### 4.4 Comptime evaluator
|
||||
|
||||
An interpreter over HIR subgraphs. Eligible only if: no side effects, primitives only, no global mutation, no I/O, and within loop/recursion budgets. Otherwise it bails and defers to VIR. Outputs `const_value`, then `is_comptime` + `is_constant`.
|
||||
|
||||
The frontend `static` attribute **seeds** the evaluator. HIR trusts `static` as "this is compile-time" but still validates it.
|
||||
|
||||
**Budgets (configurable; defaults):**
|
||||
|
||||
| Budget | Default |
|
||||
|---|---|
|
||||
| `comptime.max_iterations` | 4096 |
|
||||
| `comptime.max_recursion_depth` | 256 |
|
||||
| `comptime.max_steps` | 1,000,000 |
|
||||
| `comptime.max_aggregate_elements` | 65,536 |
|
||||
| `hir.max_rewrites_per_function` | 100,000 |
|
||||
| `hir.max_rewrites_per_module` | 1,000,000 |
|
||||
|
||||
All are overridable via the API and CLI.
|
||||
|
||||
### 4.5 Defer
|
||||
|
||||
HIR carries a real high-level `defer` statement. The optimizer inlines the deferred call at every scope-exit path and then erases the `defer` marker — zero runtime tax.
|
||||
|
||||
### 4.6 Optimization-level behavior
|
||||
|
||||
| Level | HIR | VIR |
|
||||
|---|---|---|
|
||||
| `-O0` | traits validated only | off |
|
||||
| `-O1` | builtin folds + discovery | off |
|
||||
| `-O2` | full cascade + comptime | **on** |
|
||||
| `-O3` | + special AST transforms (loop-unfold/vectorize → constant-unfold → comptime) | on |
|
||||
| `-Ofast` | = `-O3` + `-march=native` + fast-math | on |
|
||||
| `-Oz` | size-tuned | on |
|
||||
|
||||
**Compiler-friendly C is emitted at ALL optimization levels.** `-O3` merely adds more aggressive AST transforms.
|
||||
|
||||
---
|
||||
|
||||
## 5. Lowering (HIR → VIR)
|
||||
|
||||
**Role:** the "middleman". Pure translation + SSA construction; **no discovery**.
|
||||
|
||||
### 5.1 Contract
|
||||
|
||||
- **Input:** annotated HIR with **complete, epoch-valid** traits.
|
||||
- **Output:** VIR — basic blocks, phi-nodes, def-use, SSA.
|
||||
- Lowering **validates trait completeness**. A missing or contradictory trait raises an **internal-compiler-error** diagnostic (see §11).
|
||||
- Deterministic; parallelizable per function.
|
||||
|
||||
### 5.2 Form
|
||||
|
||||
Lowering is a **streaming recursive interpreter** over the HIR tree ("tcc-for-VIR"):
|
||||
|
||||
- Structured HIR (`if`/`while`/`for`/`switch`/`block`) → blocks + terminators (`br`/`condbr`/`switch`/`ret`/`unreachable`).
|
||||
- Expressions → temporaries + instructions.
|
||||
|
||||
### 5.3 SSA construction
|
||||
|
||||
SSA is built with the **Braun et al. sealed-block algorithm**: one pass, on-the-fly phi insertion, no separate dominance-frontier pass. A loop header is sealed once its back-edge is emitted.
|
||||
|
||||
### 5.4 Memory model (trait-driven hybrid)
|
||||
|
||||
| Condition | Representation |
|
||||
|---|---|
|
||||
| scalar ∧ ¬`is_addressed` ∧ ¬`escapes` ∧ ¬`is_runtime_mutable` | pure SSA value |
|
||||
| addressed / escaping / runtime-mutable / aggregate | `alloca` + explicit `load`/`store` |
|
||||
| volatile / atomic | forced memory, never promoted |
|
||||
|
||||
VIR's `mem2reg`/`SROA` MAY promote memory back to SSA when traits confirm safety.
|
||||
|
||||
### 5.5 Phi
|
||||
|
||||
VIR has an explicit LLVM-style `Phi` instruction whose operands are `(value, predecessor-block)` pairs.
|
||||
|
||||
### 5.6 Identity & traits
|
||||
|
||||
A `LoweringMap` (HIR node → VIR entity(ies)) is built during lowering. Each VIR entity carries its own `Trait`, populated from the source HIR node. HIR Relationship endpoints are rewritten to VIR entities via the map. VIR depends only on `vct.traits` + node identity.
|
||||
|
||||
### 5.7 Optimization barriers
|
||||
|
||||
`complex`/`NoOptimize` nodes are lowered inside a **region-level** optimization barrier. VIR passes MUST NOT rewrite across it.
|
||||
|
||||
### 5.8 Defer fallback
|
||||
|
||||
If HIR optimization is off (`-O0`/`-O1`), lowering itself expands `defer` at every scope exit.
|
||||
|
||||
### 5.9 VIR type system
|
||||
|
||||
More minimal than HIR but still fleshed out: integers with width, floats, pointers, aggregates, function types, `void`.
|
||||
|
||||
### 5.10 Output invariants
|
||||
|
||||
Well-formed SSA:
|
||||
|
||||
- every use is dominated by its def,
|
||||
- every block is terminated,
|
||||
- phi arity equals predecessor count,
|
||||
- a single def per value.
|
||||
|
||||
Verified in debug builds (§12).
|
||||
|
||||
---
|
||||
|
||||
## 6. VIR Optimizer
|
||||
|
||||
**Role:** the heavy-lifting half. Reads traits only, transforms SSA, performs **no discovery**. Runs at `-O2`/`-O3`/`-Ofast`/`-Oz`.
|
||||
|
||||
### 6.1 Contract
|
||||
|
||||
- **Input:** lowered VIR (well-formed SSA/CFG) + complete traits.
|
||||
- **Output:** optimized VIR, still in SSA.
|
||||
- **Strict trait consumer:** a transform is enabled by an attribute/relationship or requested by a Request. VIR MUST NOT assume a fact absent from traits.
|
||||
|
||||
### 6.2 Framework
|
||||
|
||||
```d
|
||||
interface VirPass {
|
||||
string name();
|
||||
void run(VirModule m, PassContext ctx);
|
||||
}
|
||||
```
|
||||
|
||||
**Analyses:** `DominatorTree`, `PostDominatorTree`, `LoopInfo`, `AliasInfo`, `DefUse`, `CallGraph`, `RangeInfo`. Passes declare which analyses they preserve; the manager invalidates the rest (**fine-grained**, not invalidate-all). Epoch counters catch stale trait reads.
|
||||
|
||||
### 6.3 Default pipeline (`-O2`)
|
||||
|
||||
A module pass manager interleaving function passes and IPA:
|
||||
|
||||
1. **Canonicalize:** `mem2reg`, `SROA`, `instcombine`, `simplifyCFG`, `early-CSE`, `DCE`.
|
||||
2. **Scalar:** `GVN`, `SCCP`, `LICM`, `indvars`, `reassociation`.
|
||||
3. **IPA:** inliner (driven by `Inline`/`AlwaysInline`/`NoInline`/`ColdCall`/`LikelyCall`), global `DCE`, `IPSCCP`, function-attribute propagation.
|
||||
4. **Loops:** `unroll`/`interleave`/`peel`/`rotate`/`unswitch`/`distribute`/`fuse`/`jam`, each gated by its Request.
|
||||
5. **Vectorize:** loop + SLP, gated by `Vectorize`/`LoopInvariant`/`Range`/`NoAlias`/`Restrict`.
|
||||
6. **Memory:** alias-driven `DSE`, GEP simplification, load widening.
|
||||
7. **Control flow:** block layout (`HotPath`/`ColdPath`), tail-call formation (`TailCall`/`MustTail`/`NoTail`), unreachable pruning, jump threading.
|
||||
8. **Codegen prep:** canonicalize for the backend; remains SSA.
|
||||
|
||||
`-O3`/`-Ofast` raise aggression. `-Oz` is size-first (most unrolling/vectorization off).
|
||||
|
||||
### 6.4 Requests
|
||||
|
||||
Each Request yields a `RequestResult` (§3.2). Denials carry a `DenyReason` + one-line note.
|
||||
|
||||
### 6.5 Alias analysis
|
||||
|
||||
Synthesized from `NoAlias`/`Restrict`/`NoCapture`/`Dereferenceable`/`ReadOnly`/`WriteOnly` + provenance — no guessing.
|
||||
|
||||
### 6.6 Barriers
|
||||
|
||||
Region optimization barriers (`complex`/`NoOptimize`, §5.7) are opaque; passes skip across them.
|
||||
|
||||
### 6.7 Budgets
|
||||
|
||||
`vir.max_iterations`, `vir.max_pipeline_rounds` (configurable).
|
||||
|
||||
### 6.8 Resolutions
|
||||
|
||||
- A **Strong-request denial is a hard error** when the backend is capable and the reason is `Illegality`/`ContradictsTrait`; it is a **warning** for `Unsupported`/`NoTargetSupport`.
|
||||
- The pipeline is a **fixed canonical pipeline** (LLVM-style), not adaptive request-driven ordering.
|
||||
- **Inlining is a VIR/IPA transform** (HIR only requests it).
|
||||
- `RangeInfo` is **lightweight**, seeded by `Range`/`Assume` traits — no full ScalarEvolution.
|
||||
- Analysis invalidation is **fine-grained** (preserved-analyses sets).
|
||||
|
||||
---
|
||||
|
||||
## 7. C Backend (`vct.backend.c`)
|
||||
|
||||
**Role:** VIR → C17.
|
||||
|
||||
### 7.1 Contract
|
||||
|
||||
- **Input:** optimized VIR (still SSA, traits attached).
|
||||
- **Output:** one C17 translation unit **per function**.
|
||||
- Emission is per-function independent → parallel; the driver concatenates deterministically.
|
||||
- Two-pass emission: forward declarations, then definitions.
|
||||
|
||||
### 7.2 Out-of-SSA
|
||||
|
||||
Boissinot et al., *"Revisiting Out-of-SSA Translation"* — handles the lost-copy and swap problems. **Critical edges are split before** out-of-SSA.
|
||||
|
||||
### 7.3 Compiler-friendly C
|
||||
|
||||
- **Structured control-flow reconstruction** (`if`/`else`/`while`/`for`/`do`/`switch`) for reducible CFGs; `goto` fallback for irreducible.
|
||||
- **Qualifiers from traits:** `restrict` (`Restrict`/`NoAlias`), `const` (`ReadOnly`/`NoWrite`), `_Noreturn` (`NoReturn`), `cold`/`hot` (`ColdPath`/`HotPath`), `static inline` / `noinline`.
|
||||
- **Hints:** `__builtin_expect`; `__builtin_assume_aligned` / `_Alignas` (`Align`); `__builtin_unreachable` (`Unreachable`); `__builtin_assume` (clang, `Assume`); `#pragma GCC ivdep` / `#pragma clang loop vectorize(enable)` (`Vectorize`); `pure`/`const` (`NoRead`/`NoWrite`).
|
||||
- **`NoOptimize`/`complex` barrier:** a compiler fence — `__asm__ __volatile__("" ::: "memory")` plus an opaque `noinline` call.
|
||||
|
||||
### 7.4 Types & layout
|
||||
|
||||
`intN_t`/`uintN_t`, `float`/`double`, `T*`, `struct`/`union`/arrays, function pointers, `void`. VIR aggregate GEPs use **field indices**, not byte offsets; the C compiler picks layout. An explicit **layout trait** pins the ABI (emits padding + `_Static_assert`).
|
||||
|
||||
### 7.5 Naming
|
||||
|
||||
Deterministic mangling. Exported names preserved; internal names sanitized and uniquified. A reserved-word/collision prefix table.
|
||||
|
||||
### 7.6 Intrinsics
|
||||
|
||||
Mostly `__builtin_*` (GCC/Clang), plus a small set of portable fallback helpers for what builtins do not cover.
|
||||
|
||||
### 7.7 Debug mapping
|
||||
|
||||
`#line` directives back to frontend source, flag-gated (on for debug builds, off for release). The frontend registers its source map with `vct.diag`.
|
||||
|
||||
### 7.8 ABI
|
||||
|
||||
Exported functions match the C ABI; internal functions are `static`; struct-by-value follows the C compiler's ABI. `MustTail` requires `[[clang::musttail]]` on Clang; on GCC a `MustTail` request is a **hard error**.
|
||||
|
||||
---
|
||||
|
||||
## 8. LLVM Backend (`vct.backend.llvm`)
|
||||
|
||||
**Role:** VIR → textual LLVM IR, **post** VIR optimization. Pure translation; no optimization or discovery. Output is a self-contained `.ll` module. libVCT never invokes `llvm-as`/`opt`/`llc` or links LLVM — that is the frontend's job. Deterministic per-function emission with stable concatenation.
|
||||
|
||||
### 8.1 SSA mapping
|
||||
|
||||
VIR is already SSA → **1:1 mapping**: instruction → instruction, block → block, `Phi` → `phi`, terminators, `alloca`/`load`/`store`. **No out-of-SSA** (the opposite of the C backend).
|
||||
|
||||
### 8.2 Types
|
||||
|
||||
`iN` integers, `float`/`double`, **opaque `ptr`** (not typed), `struct`/array (VIR field indices → LLVM struct indices), `void`, function types.
|
||||
|
||||
### 8.3 Traits → LLVM attributes/metadata
|
||||
|
||||
| Trait | LLVM |
|
||||
|---|---|
|
||||
| `Restrict`/`NoAlias` | `noalias` + `!alias.scope`/`!noalias` |
|
||||
| `ReadOnly`/`WriteOnly`/`NoRead`/`NoWrite` | `readonly`/`writeonly` |
|
||||
| `NonNull` | `nonnull` |
|
||||
| `Align(n)` | `align n` |
|
||||
| `Range` | `!range` on loads |
|
||||
| `Assume` | `llvm.assume` |
|
||||
| `NoInline`/`AlwaysInline` | `noinline`/`alwaysinline` |
|
||||
| `TailCall`/`MustTail`/`NoTail` | `tail`/`musttail`/`none` |
|
||||
| `NoReturn` | `noreturn` |
|
||||
| `HotPath`/`ColdPath` | `!prof` |
|
||||
| `Vectorize` | `!llvm.loop.vectorize.enable` |
|
||||
| `Unroll(n)` | `!llvm.loop.unroll.count` |
|
||||
| `Unreachable` | `unreachable` |
|
||||
| `NoUndef` | reserved; only meaningful here |
|
||||
|
||||
### 8.4 Intrinsics
|
||||
|
||||
`memcpy`/`memset`/`memmove`, `llvm.sqrt.*`, `llvm.fabs.*`, `llvm.ctpop.*`, `llvm.fshl.*`, saturating/overflow ops.
|
||||
|
||||
### 8.5 Module scaffolding
|
||||
|
||||
Target triple (from `-target`) + derived target datalayout, so the module is self-contained. Declarations precede definitions.
|
||||
|
||||
### 8.6 Resolutions
|
||||
|
||||
- Pin **LLVM 18+** only (opaque pointers, current metadata).
|
||||
- **No `!llvm.dbg`** in v1; debug only on the C path via `#line`.
|
||||
- Emit **text `.ll` only**; the caller runs `llvm-as`.
|
||||
- Emit **real** `llvm.assume` / `!range`.
|
||||
|
||||
---
|
||||
|
||||
## 9. Driver & CLI
|
||||
|
||||
### 9.1 Role split
|
||||
|
||||
- **`vct.driver`** — library-level orchestrator: takes emitted C TUs, invokes the C compiler, collects `.o`. Owns the `-llvm` path (returns LLVM IR, never touches `cc`) and the `-mangled` path.
|
||||
- **`vct.cli`** — thin wrapper: parses flags into `Config`, drives the whole pipeline (read IR → HIR opt → lower → VIR opt → `backend.c` → driver), formats diagnostics, sets exit codes. GC-allocated; holds no optimizer state.
|
||||
|
||||
### 9.2 Pipeline
|
||||
|
||||
```
|
||||
read input
|
||||
-> (HIR opt if enabled)
|
||||
-> lower
|
||||
-> (VIR opt if enabled)
|
||||
-> backend.c -> driver -> .o
|
||||
```
|
||||
|
||||
`-S` stops after C; `-emit-llvm`/`-llvm` stop after LLVM IR.
|
||||
|
||||
### 9.3 Flag surface (v1)
|
||||
|
||||
| Group | Flags |
|
||||
|---|---|
|
||||
| **Optimize** | `-O0` `-O1` `-O2` `-O3` `-Ofast` `-Oz` |
|
||||
| **Target** | `-march=<arch>` (→ cflags), `-mcpu=<cpu>`, `-target <triple>` |
|
||||
| **Modes** | `-S` (C only), `-emit-llvm`/`-llvm`, `-mangled` (cosmopolitan) |
|
||||
| **Toolchain** | `-cc=<gcc\|clang>` (default auto), `-cflags="..."`, `-j<N>`, `-save-temps` |
|
||||
| **LTO** | `-flto[=full\|thin]`, `-ffat-lto-objects` |
|
||||
| **Debug** | `-g`, `--dump-hir`, `--dump-vir`, `--verify`, `--time-passes`, `--stats` |
|
||||
| **Output** | `-o <path>` |
|
||||
| **Warnings** | `-Wall -Werror -Wextra` **always on**; `-Wno-error` escape hatch; `-w` to silence |
|
||||
| **Diagnostics** | `--remarks`, `--diagnostics=json` |
|
||||
| **Pass toggles** | every pass exposed as `-f<pass>` / `-fno-<pass>` |
|
||||
|
||||
### 9.4 cc invocation
|
||||
|
||||
```
|
||||
cc -std=c17 -c <tu.c> -o <tu.o> <cflags> <march> -Wall -Werror -Wextra <g> <flto>
|
||||
```
|
||||
|
||||
One process per TU, bounded by `-j`. The compiler is discovered via `$CC` then `PATH`; its version/dialect is probed once and drives the backend's pragma/builtin choices (§7.3, §7.6).
|
||||
|
||||
### 9.5 Error handling
|
||||
|
||||
`cc` stderr is captured. `#line` directives (§7.7) map C errors back to VIR/source; they are re-emitted via `vct.diag`. A `cc` failure produces a diagnostic and a nonzero exit.
|
||||
|
||||
### 9.6 Determinism
|
||||
|
||||
Stable TU ordering and flag ordering. Temp files live under a GC-managed temp dir (cleaned unless `-save-temps`).
|
||||
|
||||
### 9.7 `-mangled`
|
||||
|
||||
Switches the toolchain to `cosmocc`/cosmopolitan libc and produces an Actually Portable Executable. This mode **does link** — the opt-in exception to "caller links".
|
||||
|
||||
### 9.8 Resolutions
|
||||
|
||||
- Driver compiles to `.o` only; a single `-o` merges per-function objects with an `ld -r` relocatable link; a recommended link line / response file is emitted for the caller. `-mangled` is the sole mode producing a final executable.
|
||||
- The CLI also reads a **textual HIR/VIR format** for testing/tools/round-trips; frontends still embed the library.
|
||||
- **Every pass** is exposed as a `-f`/`-fno` toggle.
|
||||
- `-Wall -Werror -Wextra` default; `-Wno-error` escape hatch; `-Werror` applies to libVCT-generated C.
|
||||
|
||||
---
|
||||
|
||||
## 10. Diagnostics & Source Maps (`vct.diag`)
|
||||
|
||||
### 10.1 Central service
|
||||
|
||||
One central diagnostics service; every module emits through it, nothing prints directly. GC-allocated.
|
||||
|
||||
```d
|
||||
struct Diagnostic {
|
||||
Severity severity; // Error | Warning | Note | Remark | Ice
|
||||
DiagCode code; // stable namespaced, e.g. VCT1002
|
||||
string message;
|
||||
Span primary;
|
||||
Span[] notes;
|
||||
Suggestion[] suggestions;
|
||||
}
|
||||
```
|
||||
|
||||
### 10.2 Source maps
|
||||
|
||||
The frontend registers `SourceLocation{file,line,col,len}` against HIR nodes; VIR resolves via the `LoweringMap` (§5.6). Degradation to no-location is graceful. The C backend emits `#line` (§7.7); `cc` errors are parsed and re-mapped (§9.5).
|
||||
|
||||
### 10.3 Channels
|
||||
|
||||
- `DiagnosticConsumer` callback (canonical for embedded use).
|
||||
- Human-readable CLI (color when tty).
|
||||
- JSON (`--diagnostics=json`, **versioned schema**, shipped in v1).
|
||||
|
||||
### 10.4 Optimizer remarks
|
||||
|
||||
`--remarks` (off by default; automatically enabled at `-O3`) emits per-decision remarks tied to the trait pipeline, e.g.:
|
||||
|
||||
- `HIR: replaced 3 uses of node#412 (comptime 64)`
|
||||
- `VIR: denied Inline on @foo — CostModel (callee 2.4x size budget)`
|
||||
|
||||
### 10.5 ICE policy
|
||||
|
||||
Internal compiler errors include: missing/contradictory trait at lowering (§5.1), VIR invariant violated in debug (§5.10), and a Strong request denied for `Illegality`/`ContradictsTrait` (§6.8).
|
||||
|
||||
- **Release:** return a failure result + ICE diagnostic with a module-dump request.
|
||||
- **Debug:** assert.
|
||||
|
||||
Never silently continue.
|
||||
|
||||
### 10.6 Exit codes
|
||||
|
||||
| Code | Meaning |
|
||||
|---|---|
|
||||
| 0 | success |
|
||||
| 1 | diagnostics present |
|
||||
| 2 | usage/config error |
|
||||
| 3 | ICE |
|
||||
|
||||
### 10.7 Resolutions
|
||||
|
||||
- The trait-mismatch warning **is** promoted to error under `-Werror` (frontends can opt out with `-Wno-error=`).
|
||||
- ICE in release = diagnostic + error result.
|
||||
- Remarks: both opt-in `--remarks` **and** automatically enabled at `-O3`.
|
||||
- Ship a versioned JSON diagnostics format in v1.
|
||||
|
||||
---
|
||||
|
||||
## 11. Testing & Verification
|
||||
|
||||
### 11.1 Core constraint
|
||||
|
||||
VIR is a strict trait consumer and cannot be tested without a valid HIR fixture. The test strategy is anchored on a fixture layer.
|
||||
|
||||
### 11.2 Layers
|
||||
|
||||
1. **Unit tests** — D `unittest` per module.
|
||||
2. **HIR test-builder harness** (`vct.test.hirbuild`) — a fluent builder constructing HIR modules + traits without a frontend. This is the fixture layer for HIR-opt, lowering, VIR-opt, and both backends.
|
||||
3. **Textual round-trip** — property: `print(parse(print(m))) == print(m)`.
|
||||
4. **FileCheck-style tests** — `CHECK` / `CHECK-NOT` / `CHECK-NEXT` directives.
|
||||
5. **Verifier** — structural SSA (§5.10) + trait consistency, via `--verify`; asserted after every pass in debug.
|
||||
6. **Differential** — the VIR reference interpreter vs the compiled `.o`; the C path vs the LLVM path.
|
||||
7. **Fuzz** — random HIR/trait combinations; property: terminates, verifier clean, output compiles.
|
||||
8. **Matrix** — gcc + clang × `-O0`..`-Ofast`/`-Oz` × C/LLVM.
|
||||
9. **Perf benchmarks** — codegen quality is the headline; compile time tracked via `--time-passes`/`--stats`.
|
||||
|
||||
### 11.3 VIR reference interpreter
|
||||
|
||||
Approved as **v1 scope**. It is the differential oracle: the same VIR is run through the interpreter and through the compiled `.o`, and results are compared.
|
||||
|
||||
### 11.4 Determinism
|
||||
|
||||
Every pipeline run is executed twice in CI; output must be byte-identical.
|
||||
|
||||
### 11.5 Regression corpus
|
||||
|
||||
A corpus of input IR + expected FileCheck patterns; CI runs the full matrix.
|
||||
|
||||
### 11.6 Resolutions
|
||||
|
||||
- FileCheck-style matcher (not exact golden files).
|
||||
- Build the VIR reference interpreter in v1 as a differential oracle.
|
||||
- Verifier always-on in debug builds; `--verify` opt-in in release.
|
||||
- Random-HIR + random-trait fuzzers in v1.
|
||||
|
||||
---
|
||||
|
||||
## 12. Build, Packaging & Public API Lifecycle
|
||||
|
||||
### 12.1 Language & toolchain
|
||||
|
||||
Written in D. Primary compiler: **LDC2**; **DMD** supported as a secondary build target. **Zero runtime dependencies** — only a D compiler to build, and a system C compiler discovered at runtime for the C path.
|
||||
|
||||
### 12.2 Build system
|
||||
|
||||
`dub` for normal development, **plus** a plain `build.d`/Makefile path for no-dub builds. Both MUST produce identical artifacts.
|
||||
|
||||
### 12.3 Artifacts
|
||||
|
||||
- `libvct.a` (static) and `libvct.so` (shared).
|
||||
- `vct` CLI binary.
|
||||
- `vctc.h` — C API header.
|
||||
- `vct.test.hirbuild` — shipped publicly.
|
||||
|
||||
### 12.4 API surfaces
|
||||
|
||||
- **Native D API** (primary, full feature set).
|
||||
- **Thin `extern(C)` C API** (v1): opaque handles for `Context`/`Module`/`Builder`/`Config`, functions to build IR, run the pipeline, and query diagnostics. No logic lives in the shim.
|
||||
|
||||
### 12.5 Module visibility
|
||||
|
||||
- **Public:** `vct.ir.hir`, `vct.traits`, `vct.context`, `vct.backend.c` entry, `vct.backend.llvm` entry, `vct.driver`, `vct.diag`, `vct.test.hirbuild`.
|
||||
- **Internal:** `vct.ir.vir`, `vct.lower`, `vct.hir.opt`, `vct.vir.opt`, `vct.comptime` internals.
|
||||
|
||||
### 12.6 Memory ownership & lifecycle
|
||||
|
||||
- A `Context` owns one or more **arenas**; an arena is the unit of reclamation. Default: one arena per `Module`, freed wholesale.
|
||||
- All IR and optimizer objects are arena-owned and non-GC; passes mutate in place under epoch guards.
|
||||
- GC is permitted only on cold paths (driver, diagnostics, CLI).
|
||||
- `Context` accepts an **`ArenaAllocator` interface** so embedders can supply backing memory.
|
||||
|
||||
### 12.7 `Config`
|
||||
|
||||
One struct holds opt level, target (`-march`/`-mcpu`/`-target`), all budgets, pass toggles, warning policy, cc selection, and mode (`-S`/`-llvm`/`-mangled`). The CLI is a pure function from argv to `Config`; the library takes a `Config`. Programmatic callers bypass the CLI.
|
||||
|
||||
### 12.8 Threading contract
|
||||
|
||||
Thread-safe when each compilation unit has its own `Context`/arenas; no shared mutable global state. Intra-module parallelism is internal and bounded by `-j`.
|
||||
|
||||
### 12.9 Versioning
|
||||
|
||||
- Library semver.
|
||||
- **Textual IR format version** — separate and independently versioned.
|
||||
- **Trait vocabulary version** — adding attributes/requests is backward-compatible; changing semantics is a major bump. Experimental passes are gated behind feature flags.
|
||||
|
||||
### 12.10 Determinism guarantee
|
||||
|
||||
Same `Config` + same input ⇒ byte-identical output, independent of thread count.
|
||||
|
||||
### 12.11 Resolutions
|
||||
|
||||
- C API in v1 (thin shim).
|
||||
- Custom `ArenaAllocator` hooks exposed.
|
||||
- LDC2 primary + DMD secondary.
|
||||
- `dub` + no-dub build path.
|
||||
|
||||
---
|
||||
|
||||
## 13. Worked End-to-End Example
|
||||
|
||||
This example is **normative** for trait/request semantics.
|
||||
|
||||
Input program:
|
||||
|
||||
```
|
||||
fn foo(x: int, y: int) -> int {
|
||||
return x + y;
|
||||
}
|
||||
|
||||
fn main() -> int {
|
||||
x = 4;
|
||||
y = x + 4;
|
||||
z = foo(x, y);
|
||||
a = sqrt(z);
|
||||
println(a);
|
||||
}
|
||||
```
|
||||
|
||||
The HIR cascade proceeds as follows:
|
||||
|
||||
1. HIR comptime-evaluates `x = 4` → `is_comptime`, `is_constant`, `const_value = 4`.
|
||||
2. `y = x + 4` unfolds to `y = 8` → `is_comptime`.
|
||||
3. `foo(x, y)` is proven small → HIR issues an `Inline` **request**; `z` becomes `8 + ...` constant → comptime.
|
||||
4. `a = sqrt(z)` → `a = sqrt(8)` comptime → `a = 3.4641016151377544`.
|
||||
5. The whole program collapses to `println(3.4641016151377544)`.
|
||||
|
||||
VIR then optimizes the `println` call. The emitted C is effectively a single call with the folded constant. This demonstrates trait propagation HIR → VIR and the request lifecycle (`Inline` accepted).
|
||||
|
||||
---
|
||||
|
||||
## 14. Open Questions & Future Work
|
||||
|
||||
- **Debug info on the LLVM path** (`!llvm.dbg`) — deferred past v1.
|
||||
- **Windows / MSVC support** — out of scope for v1.
|
||||
- **LLVM bitcode emission** — deferred; caller runs `llvm-as`.
|
||||
- **Incremental / cached compilation** — content-hash `.o` caching is out of scope for v1.
|
||||
- **Cross-compilation targets** beyond `-target` passthrough and cosmopolitan — future work.
|
||||
Reference in New Issue
Block a user