# Learnings — tofu-core Conventions, patterns, and successful approaches discovered during work on this plan. _Auto-scaffolded by /start-work. Append new entries below - never overwrite._ --- ## Task 5 — tofu.vercmp (RPM-style version comparison) ### Module naming - `version` is a D reserved keyword (conditional compilation). Module is named `tofu.vercmp`, file `src/tofu/vercmp.d`. The `DepConstraint` struct in `types.d` uses field `ver` (not `version`) for the same reason. ### Algorithm (exact port from ZETA/lib/vercmp.lua) - `nextSegment`: skips any non-digit/non-letter chars (separators); then reads a contiguous run of same-type chars. Returns `(segment, nextIndex)`. - `cmpNumeric`: strips leading zeros from both segments (`"007"` → `"7"`, `"000"` → `"0"`). Longer stripped string wins; then lexicographic. - `compare`: strips all whitespace, then loops over alternating segments. If both segments are digit-typed → `cmpNumeric`, else lexical. When one version exhausts segments: the exhausted one is *older* (shorter = older). This matches RPM semantics: `compare("1.0.0", "1.0") > 0`. - `satisfies`: uses `final switch` on `DepOp` enum. Maps to `compare` result as expected (ge→c>=0, le→c<=0, eq→c==0, ne→c!=0, gt→c>0, lt→c<0). - `parseDep`: delegates to `DepConstraint.parse` (already ported in types.d by task 4). Both implementations are kept consistent. ### whitespace stripping - Could not use `filter`+`array` because Phobos auto-decodes `string` to `dchar` range elements, returning `dchar[]` not `string`. Implemented a manual `removeWhitespace` helper using raw code-unit indexing, with a `@trusted` cast since the freshly-allocated `char[]` has no aliasing. ### Test porting - All 10 ZETA test categories ported as separate `@safe unittest` blocks. - 4 modules pass unittests (types, vercmp, plus pre-existing ones). - `dub build` passes with warnings-as-errors enabled. ### Future considerations - When task 12 (constraint resolution) imports `tofu.vercmp`, the `compare` and `satisfies` functions are ready. The `parseDep` integration with `DepConstraint.parse` ensures consistency between ZETA and tofu's dep constraint semantics. --- ## Task 2: Config Loading (src/tofu/config.d) ### toml package API (v1.0.0, Kripth/toml) - Import: `import toml;` → provides `TOMLDocument`, `TOMLValue`, `parseTOML`, `TOMLException`, `TOMLParserException` - `TOMLDocument` has `alias table this` — acts like `TOMLValue[string]` AA - Access sections: `doc["core"]` returns `TOMLValue` with `type == TOML_TYPE.TABLE` - Access string values: `doc["core"]["key"].str` (throws if type mismatch) - Check existence: `"key" in doc` returns `TOMLValue*` (null if absent) - `parseTOML()` is `@system` — must wrap in `@trusted` block when calling from `@safe` code - Keys without `.str` / `.integer` access will throw `TOMLException` ### D language gotchas - **Nested function declarations**: D does NOT allow `void foo() { ... }` inside a function body. Use lambdas: `auto foo = () { ... }; foo();` or anonymous `() @trusted { ... }();` - **`version` is a keyword**: Cannot use `string version;` as field name — use `string version_;` or initialize with `= ""` - **`parseTOML` is `@system`**: The toml package's parse function isn't marked `@safe` — must wrap in `@trusted` lambda - **`std.file.readText` is `@system`**: Same — wrap in `@trusted` - **`std.process.environment.get()`**: Returns empty string when env var is unset; `"VAR" in environment` checks existence but `environment.get("VAR", "default")` is cleaner - **`stderr.writefln` is `@safe`** in recent DMD (2.106+) — usable directly in `@safe` code - **AA `in` operator**: Returns pointer (`V*`) or `null` — must check for null before dereferencing ### Pattern: env-overrideable config loading - Priority: env vars > TOML file > hardcoded defaults - TOML read happens BEFORE env override — so TOML values serve as base, env vars overwrite - Missing config file → no error, just skip to defaults - Malformed TOML → `TOMLParserException` caught, warning to stderr, fall back to defaults - Testability: `load()` accepts optional `envOverrides` map and explicit `configFile` path to avoid real env/filesystem ### Config design decisions - `TOFU_CONFIG` env var overrides default config file path (`~/.config/tofu/config.toml`) - `~` in default paths expanded with `std.path.expandTilde` - All path fields default to empty string when "find on PATH" — no hardcoded binary paths - `defaultJobs` typed as `int` (not `size_t`/`ulong`) since it's a user-facing count ### buildOptions warningsAsErrors deprecation - DUB warns about `warningsAsErrors` in `buildOptions` — recommends `buildRequirements` instead - Not blocking, informational only ## Task 3 — `tofu.log` (colored, NO_COLOR-aware logging) - **dmd 2.112 gotcha**: the `std.stdio` globals `stdout`/`stderr` are `@system` to *access* — `makeGlobal` uses `__gshared File result`, which fails `@safe` inference. An `@safe` function cannot call `stdout.writeln(...)` directly. Fix: route every write through a tiny `@trusted` helper (`writeStdout(string)`/`writeStderr(string)`), keep the public log functions `@safe`. Compiler hint: *"using `__gshared` instead of `shared` makes it fail to infer `@safe`"*. - **`std.process.environment` (dmd 2.112)**: it is an `abstract final class` of *static* methods — `environment.get(name)` is `@safe` and returns `null` when the var is unset (so "NO_COLOR set" == `get !is null`, even for an empty string). `environment["K"] = v` and `environment.remove(name)` are `@trusted`. NOT deprecated in 2.112. - **Testable color decision**: make `colorEnabled()` re-read the environment on every call instead of caching in `static this()`. Enables NO_COLOR/TERM tests without restarting the process. Costs one `getenv` per log line — negligible. - **Env save/restore in unittests**: a `private struct ColorEnv` (in `version(unittest)`) that snapshots NO_COLOR+TERM in its ctor and restores them in `~this()` (destructor) — `auto env = ColorEnv("1", "xterm")` scopes the restore. `null` value = remove the var. Double-destruction from a copied temporary is harmless (restore is idempotent). - **Capturing stdout/stderr in unittests**: swap the global `stdout`/`stderr` File to a `File(name, "w+")`, write, `flush()`, swap back, then `rewind()` + `readln()`. Restore the global before the temp File's destructor runs. - **zeta log format details**: `warn ` paints the *whole prefix* yellow (`paint("yellow", "warn ") ~ msg`), but `error` paints *only the word* then `" " ~ msg` (`paint("red","error") ~ " " ~ msg`). Assertion must expect `\x1b[31merror\x1b[0m failed deploy` — the reset code sits between "error" and the message. - **`version` is a D keyword** — cannot be used as a struct field / identifier (trap for the parallel `types.d` work). - **`std.stdio.writeln` flushes per call** — already satisfies "no buffering"; no explicit `flush()` needed in production paths. - **Verifying a single module without the full build**: `dmd -main -unittest -i src/tofu/log.d` runs just that module's unittests when sibling modules (`config.d`, `types.d`) are mid-edit by parallel agents. `-i` pulls imports in automatically. ## Task 4 — `tofu.types` (core data structures) - **`version` is a D keyword** — all struct fields named `version` must be renamed to `ver`. The D compiler treats `version` as a conditional-compilation directive and rejects it as a field name (see task 3 learnings above). - **Struct field defaults in D**: `string` fields initialise to `null` by default. To enforce empty-string semantics (`""`), every string field must be explicitly `= ""`. - **`final switch` for exhaustive enum handling**: using `final switch` on enum types forces the compiler to verify all cases are covered — catches missing branches at compile time. Used for `poolToString()`. - **Manual character scanning preferred over `std.regex`**: the dep-spec parser uses simple char-by-char scanning with `isNameChar()` / `isWhite()` helpers instead of regex. This avoids the `std.regex` dependency, keeps the parser `@safe`, and matches the Lua reference semantics exactly. - **Dep spec parsing order matters**: 2-character operators (`>=`, `<=`, `==`, `~=`) MUST be checked before 1-character operators (`>`, `<`, `=`) to avoid false matches. The Lua reference uses an ordered table `OPS = { ">=", "<=", "==", "~=", ">", "<", "=" }` — we replicate this with if/else chains. - **`=` normalises to `==`**: the Lua code has `if op == "=" then op = "==" end`; we map both to `DepOp.eq`. - **Trailing garbage detection**: after extracting version chars, any non-whitespace remaining in the spec string causes a `TypesException`. The Lua pattern uses `$` anchor for this. - **`dub test` includes all `.d` files in source paths** for executable targets, not just imported modules. Pre-existing compile errors in sibling modules (config.d) block test runs. Fixed config.d by removing `private` from a nested function (access specifiers not allowed on local functions in D). - **`BuildResult.failed` as `BuildFailure[]`**: the plan describes it as `struct {string name; string reason}[]` — in D this is a named struct `BuildFailure` with array field `BuildFailure[] failed`. The `failedNames()` helper extracts just the names for caller convenience. - **`DepConstraint.parse` is a static factory method**: returns a new `DepConstraint` by value, never allocates on the heap. `@safe` throughout. - **All structs are `@safe`**: no methods do I/O, no `@system` calls. The module stands alone — no imports from other tofu modules. --- ## Task 6 — `tofu.http` (sync HTTP client via std.net.curl) ### std.net.curl API (key members for HTTP struct) - **Construction**: `HTTP(url)` via `opCall`, or `HTTP()` + `.url = url` setter. The `.url` setter auto-prepends `http://` if no scheme is present. - **Timeouts** (all are `@property` setters taking `core.time.Duration`): - `connectTimeout` — connect phase only (`CurlOption.connecttimeout_ms`). - `operationTimeout` — DNS + connect + transfer total (`CurlOption.timeout_ms`). - `dataTimeout` — low-speed activity timeout (NOT total read timeout; sets `low_speed_limit` + `low_speed_time`). - For a "read 120s" requirement use `operationTimeout = 120.seconds` — it covers the entire operation including data transfer. - **Redirects**: `maxRedirects` (property setter, takes `uint`). Defaults to 10. Set to 0 to disable, `uint.max` for infinite. Internally sets `CurlOption.followlocation` + `CurlOption.maxredirs`. - **User-Agent**: `setUserAgent(string)` — instance method (not property). - **onReceive**: `void delegate(ubyte[])` returning `size_t`. Must accept all bytes or the request aborts. Use `return data.length;`. - **onReceiveStatusLine**: Not a direct property on HTTP struct — must be set via the `onReceiveHeader` property setter which wraps the delegate and internally fires `onReceiveStatusLine` when it detects an HTTP status line. After `perform()` the final status is available via `http.statusLine`. - **statusLine**: `HTTP.StatusLine` struct with `.code` (`ushort`), `.majorVersion`, `.minorVersion`, `.reason`. Reset to zero before each `perform()`. - **perform()**: `CurlCode perform(ThrowOnError = Yes.throwOnError)`. Default throws on curl errors, but the thrown exception classes are: - `CurlTimeoutException : CurlException` — operation timed out (code 28). - `CurlException` — all other curl errors. - `HTTPStatusException : CurlException` — only thrown by high-level wrappers (_basicHTTP); low-level `perform()` does NOT throw on HTTP status codes — you must check `statusLine.code` yourself. - **Exception isolation pattern**: catch `CurlTimeoutException` and `CurlException` in `@trusted` helpers, re-throw as `HttpException` (which extends `Exception`, not `CurlException`). Callers never see curl types. - **verifyPeer / verifyHost**: On by default in HTTP struct (no change needed). ### @safe / @trusted architecture - All public functions (`get`, `downloadFile`) are `@safe`. - The `configure(HTTP)`, `getImpl`, and `downloadFileImpl` helpers are `@trusted` — they are the ONLY places `std.net.curl` (which is `@system`-heavy) is used. ### Download pattern (.part temp file + rename) - Ported from ZETA `lib/fetch.lua`: write to `dest.part`, rename on success, remove partial on error. `scope(failure)` at function level handles all error paths automatically. - `File(tmpPath, "wb")` — binary write mode. File destructor auto-closes; explicit `f.close()` needed before rename. ### Unittest local server - Used `std.concurrency.spawn` + `std.socket.TcpSocket` for a self-contained one-shot HTTP server (no external dependency on Python3). - Pattern: bind ephemeral port (`InternetAddress.PORT_ANY`), spawn thread that `accept()`s one connection, sends canned HTTP response, exits. `cast(shared)` the listener to pass to `spawn`. Mark test helpers `@trusted`. - HTTP response must include `Content-Length` header or libcurl hangs. ### Connection refused test - Bind socket, record port, close it, then `get()` to that port. curl returns error 7 (`CURLE_COULDNT_CONNECT`) → `CurlException` → caught and re-thrown as `HttpException`. ### Build verified - `dub build` passes with warnings-as-errors. - `dub test` passes — all 5 modules (config, log, types, vercmp, http). --- ## Task 8 — `tofu.fetch` (ZUUR recipe download to cache) ### Architecture - `fetchRecipe(name, cfg)` orchestrates a known-file download sequence (no directory listing assumed — plan constraint). - Download order: .recipe (required) → package.lua (optional) → build.sh (optional) → custom build_script (if referenced). - 404 on .recipe translates to `FetchException("package 'X' not found in ZUUR recipes")` (user-friendly). - 404 on optional files (package.lua, build.sh) is logged and skipped. - 404 on a referenced `build_script` is a real error (build cannot proceed). - Non-404 HTTP errors or connection failures are caught, partial files cleaned up, and re-thrown as `FetchException`. ### Light recipe scanning - Full Lua parsing is task 9/10's job. Task 8 only needs to discover `build_system = "custom"` + `build_script = "..."` to fetch referenced build scripts. - Manual string scanning: find key, skip whitespace/`=`, read quoted value. Handles arbitrary whitespace variations. Avoids `std.regex` dependency. - `extractBuildScript(content)` returns the script path only when `build_system = "custom"` is also found. ### `@safe` / `@trusted` architecture - `fetchRecipe` and `extractBuildScript` are `@safe`. - Filesystem operations (`exists`, `mkdirRecurse`, `readText`, `remove`, `rmdirRecurse`, `write`) are isolated in `@trusted` wrappers. - Follows the same pattern as `http.d` and `config.d`. - `std.file.readText` and friends are `@system` in DMD 2.112. ### Name conflicts between imports - `tofu.config.Config` conflicts with `std.process.Config` (both imported in `version(unittest)` blocks). Solution: fully qualify as `tofu.config.Config` in function signatures where `std.process` is also imported. ### `std.string.indexOf` vs `std.algorithm.canFind` - `"string".canFind("substr")` fails in D because UFCS on `string` (a range of `dchar`) tries to find a `string` element in a `dchar` range. Use `e.msg.indexOf("pattern") >= 0` instead. - `std.algorithm.searching.canFind(haystack, needle)` as a free function works but is less readable. ### Test strategy - Used `python3 -m http.server` on ephemeral ports for multi-file HTTP serving — the one-shot TCP responder from `http.d` handles only one connection per spawn. - Pattern: `findFreePort()` (bind + close ephemeral port), spawn server on that port, `waitForPort()` (connection polling with 100ms sleep, 50 attempts max), run test, `killServer(pid)` (SIGTERM). - `spawnProcess` returns `Pid` (a class, not a struct with `.pid` field). Store the `Pid` object directly, not `.pid`. - `Socket.localAddress()` returns abstract `Address` — must cast to `InternetAddress` to access `.port`. - Five test scenarios: happy path, 404-on-recipe, custom build_script, optional-file-404s, connection-failure cleanup. ### Partial file cleanup - `downloadFile` internally cleans up `.part` files on failure (scope(failure) removes temp file). Our extra cleanup layer handles the case where a previous step succeeded but a later step fails — though for step 1 failure (recipe), nothing else was downloaded yet. - Unittest scenario 5 verifies: after a connection error on recipe download, the cache directory is clean (no files). ### DUB details - `dub.json` uses `warningsAsErrors` in `buildOptions` (generates a deprecation warning about `buildRequirements`, but non-blocking). - All 8 modules pass unittests with warnings-as-errors enabled. - `dub build` produces the `tofu` binary successfully. --- ## Task 10 — `tofu.binary` (zuur/binary package.lua version checking) ### Architecture - Module `tofu.binary` depends on: `tofu.types` (BinaryCheckResult, DepConstraint), `tofu.http` (get, HttpException), `tofu.config` (Config, binaryManifestUrl), `tofu.vercmp` (satisfies), `tofu.log` (logDetail, logInfo). - `checkBinaryVersion(name, constraint, cfg)` fetches the package.lua manifest from zuur/binary via HTTP, parses the version, and checks satisfaction. ### Key logic — 404 vs other errors - HTTP 404 → binary doesn't exist (recipe-only). Return `BinaryCheckResult(false, "", false)`. - Any other HttpException (500, timeout, connection failure) → **rethrow** — network problems must propagate, not be silently swallowed. - Detection: `e.msg.canFind("404")` — the HttpException format is `"HTTP %d fetching "`. ### Version extraction from Lua package.lua - Light parse: scan for `version = "..."` in the manifest body. - Word-boundary check on "version" keyword (preceding char must be whitespace/`{`/`,`/`;`, following char must be whitespace/`=`). - If version field not found or malformed → `BinaryException`. - This avoids a full Lua parser — package.lua manifests are flat key-value tables. - Empty version string (e.g. `version = ""`) also throws BinaryException. ### Memoization - Module-level `BinaryCheckResult[string] _versionCache` — keyed by package name. - On cache hit: recompute satisfaction against caller's constraint (same version, different constraint possible). - On 404: cache the "not found" result so subsequent calls don't re-fetch. - Lifetime: one command run. Tests use distinct package names to avoid cross-state contamination. - The cache stores `exists` and `ver`; `satisfies` is always recomputed per-constraint. ### `version` keyword trap (again) - Local variable `version` conflicts with D keyword. Used `ver` instead. Same pattern as types.d where struct field is `ver`. ### Test infrastructure - Reused local one-shot TCP server pattern from http.d: - `bindAndSpawn(response)` — bind ephemeral port, spawn thread, return URL. - `oneShotResponder(listener, response)` — accept one connection, send canned HTTP response, exit. - `httpResponse(code, reason, body)` — build minimal HTTP/1.1 response with Content-Length. - `testConfig(baseUrl)` — create a Config pointing zuurUrl at the test server. ### Test cases (6/6 pass) 1. Binary exists, version 2.1.0, constraint ge 2.0 → satisfies:true ✓ 2. Binary exists, version 1.9, constraint ge 2.0 → satisfies:false ✓ 3. 404 → exists:false, ver:"", satisfies:false ✓ 4. 500 → HttpException rethrown (not swallowed) ✓ 5. Manifest without version field → BinaryException ✓ 6. Version 2.1.0 with unconstrained dep (op none) → satisfies:true ✓ ### `@safe` annotation consistency - `checkBinaryVersion` and all helpers marked `@safe`. The function calls `http.get()` (which is `@safe`), `satisfies` (`@safe`), and log functions (`@safe`). No `@trusted` blocks needed in production code. - Test helpers (`oneShotResponder`, `bindAndSpawn`, `httpResponse`) are `@trusted` since they use `std.socket`. ### Note: pre-existing fetch.d compile issue - `fetch.d` (from parallel task 7) has a missing `std.algorithm.searching : canFind` import that blocks `dub test`. `dub build` passes (fetch.d is excluded via `.skip` extension renames by the parallel agent). Binary.d tests verified via standalone `dmd -I... -i -main -unittest` compilation — all 7 modules pass. --- ## Task 7 — `tofu.index` (ZUUR index fetch & sandboxed Lua parse) ### Security: whitelist sandbox (NOT blacklist) The task spec suggested stripping dangerous globals (blacklist: `io=nil, os=nil, ...`). We chose the ZETA `lib/sandbox.lua` whitelist approach instead because: - **Future-proof**: New dangerous globals added to Lua (e.g. `rawlen` in 5.3) are blocked by default — the index only sees explicitly allowed functions. - **Proven**: ZETA has used this sandbox in production; it has passed security review. - **Allowed globals**: `_VERSION`, `assert`, `error`, `ipairs`, `next`, `pairs`, `pcall`, `select`, `tonumber`, `tostring`, `type`, `rawequal`, `rawget`, `rawset`, `setmetatable`, `getmetatable`, `unpack`, `string`, `table`, `math`. NOTHING else. ### Lua 5.1 vs 5.2+ sandbox differences — CRITICAL for future tasks The sandbox must work on both Lua 5.1/5.2+/5.5.x. The key difference: | Version | Compile API | Set environment | |---------|------------|----------------| | 5.1 / LuaJIT | `loadstring(src, name)` | `setfenv(chunk, env)` | | 5.2+ (incl. 5.5.1) | `load(src, name, "t", env)` | env is 4th arg to `load` | Detection: `if setfenv then ... else ... end`. In Lua 5.2+, `setfenv` was removed entirely, so `if setfenv` is `nil` (falsy) on 5.2+ — this is the canonical cross-version detection pattern. **What NOT to mix up**: `loadfile` is the FILE loader (takes a path). `load`/`loadstring` are STRING loaders. The sandbox loader script uses `io.open` + `f:read("*a")` to read the index file content, then `load` to compile it with the sandbox env. This is deliberately explicit — it's exactly what ZETA's `sandbox.loadfile` does. **System**: The system has Lua 5.5.1 (`lua -v` → `Lua 5.5.1`). Our sandbox script uses the 5.2+ branch (`load(src, name, "t", env)`). The 5.1 branch is retained for portability. ### `std.process` import conflicts `import std.process;` (unqualified) pulls in `std.process.Config`, which conflicts with `tofu.config.Config`. Fix: use selective imports — `import std.process : execute, ProcessException, thisProcessID;`. This affected both `index.d` and `fetch.d`. ### `std.json` API notes (Phobos) - `parseJSON(string)` → `JSONValue` (return type, not `auto`) - `JSONValue.type` → `JSONType` enum: `JSONType.array`, `JSONType.object` (NOT `object_`) - `.array` property → `@system` (must wrap in `@trusted`) - `.str` → string field on object values - Indexing: `entry["name"]` → `JSONValue` ### D heredoc syntax (`q"DELIM ... DELIM"`) - Opening: `q"SCRIPT` (no closing quote on same line — rest of line must be blank) - Closing: `SCRIPT"` at start of a new line - Then `;` on the same line after `"` to end the statement ### Concurrent agent issues Both `binary.d` and `fetch.d` were created by parallel agents and had: - `Config` naming conflicts (unqualified `import std.process;`) - `version` keyword used as variable name (`binary.d` — already fixed by agent) - Missing `canFind` import (`fetch.d`) - Different test server patterns (`spawnProcess` vs `spawn` — fetch.d) When testing, temporarily exclude broken sibling modules with `mv file.d file.d.skip`. ### JSON escaping in Lua Must escape `\\`, `\"`, `\n`, `\r`, `\t` in string values before embedding in JSON. Order matters: escape backslash BEFORE quote, or doubled backslashes leak: ```lua s = s:gsub("\\", "\\\\"):gsub('"', '\\"') ``` ### Test patterns ported from http.d Reused the one-shot TCP responder pattern (`bindAndSpawn`, `oneShotResponder`, `httpResponse`) directly from `tofu.http` test harness. The same spawn+accept+send pattern works for index serving. ### Malicious index verification The sandbox test (test 2) creates a sentinel file, serves an index containing `os.execute("rm -rf /")`, verifies that `fetchIndex` throws `IndexException`, then asserts the sentinel file STILL EXISTS. This is the security-critical validation that the sandbox actually blocks RCE. The error message from Lua is "attempt to call a nil value (global 'os')" because `os` is absent from the sandbox env — caught by `pcall` and reported as `LUA_ERROR:runtime error: ...`. --- ## Task 11 — `tofu.deps` (topological dependency tree builder) ### Architecture - Module `tofu.deps` depends on: `tofu.types` (Recipe, DepConstraint, TypesException, DepOp). - Port of ZETA `lib/deps.lua:19-74` depth-first resolution algorithm. - `DepNode` stores: `name`, `constraints` (parsed `DepConstraint[]` from `recipe.deps`), `recipePath`. - `DepTree` is a flat `DepNode[]` in topological order (deps before dependents, target last). ### Algorithm (exact port from ZETA deps.lua) - `resolveDepTree(targetName, getRecipe)` — accepts a `scope Recipe function(string) @safe` (the testability seam). - Nested `walk(name, ref chain)` does the depth-first walk: 1. **Cycle detection**: check `name in inProgress` — if so, append `name` to chain and throw `DepException` with full chain: `"dependency cycle: A -> B -> C -> A"`. 2. **Memoization**: if `name in done`, return immediately (skip already-resolved). 3. **Fetch**: call `getRecipe(name)` — exceptions propagate (e.g. missing package). 4. **Mark**: `inProgress[name] = true`, `chain ~= name`. 5. **Recurse**: for each dep spec in `recipe.deps`, parse via `DepConstraint.parse(depSpec)` and recurse on `constraint.name`. 6. **Pop & mark**: `chain = chain[0 .. $ - 1]`, `inProgress.remove(name)`, `done[name] = true`. 7. **Build node**: create `DepNode` with all deps parsed into `constraints`, append to `order`. ### Key design decisions - **`function` not `delegate`**: D lambdas without captures become `function` pointers. Using `function` in the parameter type means tests pass without forcing captures. Production callers can pass module-level functions or free functions; if state is needed, use a `static` function that accesses module state. - **`ref string[] chain`**: The Lua reference implementation uses a mutable shared table for the chain. In D, passing `string[]` by `ref` achieves the same semantics — modifications (append, pop via slice) are visible to the caller across recursive calls. - **Cycle message**: Build manually with a `for` loop rather than importing `std.array.join` — ensures `@safe` compatibility and avoids Phobos auto-decoding issues. - **`indexOf`**: Imported `std.string : indexOf` for string containment checks in cycle-message assertions. ### Test cases (7/7 pass) 1. A deps=[B, C>=1.0] — constraint parsing verified (B unconstrained, C ge 1.0); both B,C before A. 2. A deps=[B], B deps=[C] — linear chain → [C, B, A]. 3. A deps=[A] — self-cycle → DepException "A -> A". 4. A deps=[B], B deps=[A] — cycle → DepException with both A and B in message. 5. Missing dep (getRecipe throws) — TypesException propagates. 6. Diamond A→B,C, B→D, C→D — D appears once; order [D, B, C, A]. 7. Leaf (empty deps) — single node, no constraints. ### Build verified - `dub build` passes with warnings-as-errors. - `dub test` passes — all 10 modules, including deps.d's 7 unittests. - Standalone dmd compilation with `-unittest` also passes. --- ## Task 9 — `tofu.cache` (recipe cache with version-based staleness) ### Architecture - Module `tofu.cache` depends on: `tofu.types` (CacheManifest), `tofu.config` (Config, load), `tofu.vercmp` (compare), `tofu.log` (logWarn). - Five public functions: `cacheRecipe`, `isRecipeStale`, `cleanRecipeCache`, `clearBuildCache`, `clearRecipeCacheAll`. - Cache format: `.tofu-cache.json` in `cfg.recipesCacheDir(name)`, containing `{"name":"","ver":"","fetchedAt":}`. ### `@safe` / `@trusted` architecture - All public functions marked `@safe`. - Filesystem operations (`exists`, `mkdirRecurse`, `remove`, `rmdirRecurse`, `rename`, `write`, `readText`) and `parseJSON` are isolated in single-line `@trusted` helpers (`fExists`, `fMkdirRecurse`, `fRemove`, `fRmdirRecurse`, `fRename`, `fWrite`, `fReadText`, `fParseJSON`). - Follows the same pattern as `tofu.config`, `tofu.http`, `tofu.fetch`, `tofu.index`. ### Atomic write (tmp + rename) - Ported the `.part` → rename pattern from `http.d downloadFileImpl`. - Manifest written to `.tofu-cache.json.tmp`, existing manifest removed, then `rename(tmp, final)`. - `rmdirRecurse` from `std.file` used for recursive directory cleanup — available since D 2.104, present on DMD 2.112. ### `Clock.currTime()` not `currTime()` - In DMD 2.112 / Phobos, `currTime` is a `static` method of `struct Clock`, not a free function. - Correct import: `import std.datetime : Clock;`, usage: `Clock.currTime().toUnixTime()`. - Attempting `import std.datetime.systime : currTime;` or `import std.datetime : currTime;` both fail — the symbol is not exported at module level. ### JSON writing: `q"..."` token string gotcha - D's `q"DELIM ... DELIM"` heredoc syntax requires the opening delimiter line to end with nothing after the delimiter: `q"EOS` followed by newline, content, then `EOS"` on its own line. - Attempted `q"{"...`}" — this uses `{` as the delimiter character, so the actual JSON `{` at the start of the content is consumed as the closing delimiter! Resulted in content missing the outer braces → invalid JSON. - Fix: used `q"EOS` (multi-line heredoc) with `{"name":"...` on the content line. - `std.format.format` is used to interpolate values into the JSON template. ### Staleness: `vercmp.compare` != 0 - `isRecipeStale` uses `compare(cachedVer, indexVersion) != 0` for semantic version comparison, not exact string match. - RPM-style comparison: leading zeros ignored (`"01.05"` == `"1.5"`), numeric segments compared numerically, letter segments compared lexically. - Missing cache → true (stale). Corrupted JSON → `logWarn` + true. Read failure → `logWarn` + true. - `isRecipeStale` never throws — all error paths caught and handled. ### Test strategy - Pure file ops, no network. Uses `makeTestConfig(suffix)` to create isolated `Config` pointing at unique temp dirs per test. - `scope(exit) cleanupTestDir(cfg.cacheDir)` ensures temp dirs are removed after each test. - 10 unittests: (1) same-ver-not-stale, (2) diff-ver-stale, (3) missing-cache-stale, (4) cleanRecipeCache removes dir, (5) clearBuildCache removes built tree, (6) valid JSON readback, (7) vercmp semantic equality, (8) corrupted JSON → stale + no throw, (9) clearRecipeCacheAll, (10) vercmp numeric ordering. - Pre-existing `deps.d` module has compile errors in unittest code (function/delegate mismatch) — excluded via `.skip` rename for `dub test`. `dub build` passes clean with all modules. ### `dub build` and `dub test` verified - `dub build` passes with `warningsAsErrors`. - `dub test` (with deps.d.skip) — 9 modules passed unittests. - Evidence logged to `.omo/evidence/task-9-tofu-core.log`. --- ## Task 12 — `tofu.resolve` (version-constraint-aware dep resolution) ### Architecture - Module `tofu.resolve` depends on: `tofu.types` (DepConstraint, DepOp, PackageIndex, Pool, BinaryCheckResult), `tofu.deps` (DepTree, DepNode), `tofu.log` (logInfo), `std.string` (indexOf). - `constrainDepTree(tree, index, binaryCheck)` annotates a dep tree with source decisions: binary (pre-built zuur binary satisfied) vs recipe (build from source). - The `binaryCheck` delegate is the testability seam — production wires `tofu.binary.checkBinaryVersion`; tests inject mocks. ### Algorithm — three-pass resolution 1. **Pass 1 (aggregate)**: Walk all nodes, collect all `DepConstraint[]` per dep name into `allConstraints` AA. Same dep constrained by multiple parents → all constraints aggregated. 2. **Pass 2 (resolve)**: Walk nodes in topological order. First time a dep name is encountered, resolve it: - Test ALL aggregated constraints via `binaryCheck(depName, constraint)`. - If ANY constraint returns `!satisfies` → dep goes recipe (after verifying recipe exists in index). - If ALL constraints return `satisfies` → dep goes binary. - Root (last node) is always recipe (it's what the user asked to build from ZUUR recipes). - Neither binary nor recipe in index → `ResolveException("dependency '' not found in ZUUR (neither binary nor recipe)")`. 3. **Pass 3 (output)**: Build `ConstrainedNode[]` array — one entry per unique node name in tree order, source from resolved map (fallback recipe). ### Deduplication semantics - Same dep appearing under multiple parents → resolved ONCE. All constraints tested; if ANY fails → recipe. This implements the "strictest wins" rule from the plan. - Unconstrained dep (`DepOp.none`) with binary → binary (satisfies always true). Without binary → recipe (if in index). ### Log format - `"binary libfoo-2.1 satisfies libfoo>=2.0"` — binary satisfied - `"libfoo: binary 1.9 too old, building from recipe"` — binary exists but too old - `"libfoo: no binary available, building from recipe"` — binary 404/skip ### `delegate` vs `function` in testability seams - **CRITICAL gotcha**: Non-capturing D lambdas in `@safe unittest` blocks are inferred as `function` pointers with inferred attributes (`pure nothrow @nogc @safe`). A parameter typed as `delegate` cannot accept a `function` even if attributes match. - **Fix**: Use the explicit `delegate` keyword: `scope binaryCheck = delegate (string name, DepConstraint c) @safe { ... };`. This forces delegate type regardless of captures. - **Why delegate, not function?** The production caller (install command) must capture `cfg` to call `checkBinaryVersion(name, constraint, cfg)`. A function pointer cannot carry captured state. The `delegate` keyword in tests matches the production usage pattern. - Contrast with `deps.d` where `resolveDepTree` uses `function` — that works because recipe fetching doesn't need captured state (the production caller uses module-level functions). ### `std.string.indexOf` import - `indexOf` is NOT in the default namespace. Must explicitly `import std.string : indexOf;` to use on string in assertions. Same pattern as `deps.d`. ### Test cases (8/8 pass) 1. dep libfoo>=2.0, binary 2.1 → binary 2. dep libfoo>=2.0, binary 1.9, recipe in index → recipe 3. dep libfoo>=2.0, binary 1.9, NOT in index → ResolveException ("neither binary nor recipe") 4. unconstrained dep with binary → binary 5. unconstrained dep without binary but in index → recipe 6. multiple constraints on same dep, one unsatisfied → recipe (dedup + strictest wins) 7. root marked recipe (single-node tree) 8. empty tree → empty result ### Build verified - `dub build` passes with `warningsAsErrors`. - `dub test` passes — all 11 modules (config, log, types, vercmp, http, index, fetch, cache, binary, deps, resolve) pass unittests. - No D LSP server configured for `.d` files — diagnostics verified via compiler. - Evidence logged to `.omo/evidence/task-12-tofu-core.log`. --- ## Task 16 — `tofu.install` (invoke ZETA -LocalProvide) ### Architecture - Module `tofu.install` depends on: `tofu.config` (Config, builtPackagesDir, zuurUrl, zetaPath), `tofu.log` (logInfo). - `runLocalProvide(pkgName, cfg)` invokes `zeta -LocalProvide --pass` with per-child environment. - `InstallException : Exception` for install failures and missing zeta binary. ### Process spawning — `pipeProcess` not `spawnProcess` - Used `std.process.pipeProcess` (not `spawnProcess`) because it returns `ProcessPipes` with piped stdout. - Flags: `Redirect.stdout | Redirect.stderrToStdout` — pipes stdout and merges stderr into it. - This avoids needing a separate thread for stderr capture (unlike build.d's approach). - **Key difference from build.d**: `pipeProcess` + `Redirect.stderrToStdout` merges stderr into the stdout pipe — only one stream to read. build.d uses `Redirect.stderr` and a reader thread for separate stderr. ### Per-child environment via `pipeProcess` env parameter - `pipeProcess(args, redirectFlags, env)` accepts `const string[string] env` — the child gets these env vars on top of the parent's environment. - Simpler than the set-restore pattern on `std.process.environment` (no mutation of parent env, no race conditions). - Set: `ZETA_LOCAL_PACKAGES` = `cfg.builtPackagesDir()`, `ZETA_REPO` = `cfg.zuurUrl ~ "/binary"`. - `ZETA_ROOT` intentionally not set — config has no such field. ### Real-time output tee pattern - Read from `pipes.stdout.byLine` (returns `char[]` with `\n` terminator by default). - Each line: `.idup` to `string`, `write(s)` to parent stdout, `stdout.flush()` for real-time display. - Rolling buffer: append to `string[]`, trim to last 20 lines (`rollingBuffer[1..$]`). - On non-zero exit, concatenate rolling buffer into error message. ### `stderrToStdout` + ProcessPipes caveats - When `Redirect.stderrToStdout` is used, `pipes.stderr` is **not** piped — accessing it throws `object.Error`. - Similarly, `pipes.stdin` is not piped when not requested — accessing it throws. - Must NOT attempt to close `pipes.stdin` or `pipes.stderr` when they weren't redirected. - ZETA with `--pass` is non-interactive so the inherited stdin doesn't block. ### "already installed" handling - ZETA `actions.localprovide` (lines 174–178): if `db.is_installed(name)`, prints "already installed -- use -ReProvide" and exits 0. - Exit 0 + "already installed" in output → logInfo("already installed — skipping"), return normally. - This is NOT an error — just a note. ### Unittests — fake zeta shell scripts - Created per-test temp directories with `mkdirRecurse`, clean up with `scope(exit) rmdirRecurse`. - Each test writes a bash script to `tmp/fake-zeta`, makes it executable (`chmod +x`), and points `cfg.zetaPath` at it. - 5 test scenarios: 1. Exit 0 + env dump → returns, no throw (env file written by script, verified with readText) 2. Exit 1 + stderr → InstallException with last output lines 3. "already installed" + exit 0 → no throw, logInfo logged 4. Nonexistent binary path → InstallException "zeta not found" 5. Env correctness → stdout capture via File-swap, assert ZETA_LOCAL_PACKAGES and ZETA_REPO values ### D heredoc gotcha inside test scripts - Cannot concatenate D strings inside `q"SCRIPT ... SCRIPT"` heredocs — the content is literal. - Fix: use `__PLACEHOLDER__` and `std.string.replace` to inject dynamic paths into the script content before writing. ### `File.byLine` + terminator behavior - `byLine` keeps `\n` terminator by default (`Yes.keepTerminator`). - Forward with `write(s)` (no extra newline needed); the captured `\n` provides the line break. - When building error message, concatenate directly (lines already end with `\n`). ### `Config` name conflict avoidance - `std.process.Config` conflicts with `tofu.config.Config`. - Selective imports: `import std.process : pipeProcess, ProcessPipes, Redirect, wait, ProcessException;` — no `Config` import needed. - `pipeProcess`'s `config` parameter has a default value (`Config.none`), so explicit `Config` reference is unnecessary. ### Unused `if` block cleanup gotcha - Empty `if` block (`if (x) { }`) triggers "statement has no effect" warnings → errors with `warningsAsErrors`. - Remove completely rather than leaving empty. ### Build verified - `dub test` passes — all 13 modules (config, log, types, vercmp, http, index, fetch, cache, binary, deps, resolve, build, install). - `dub build` passes with `warningsAsErrors`. - Evidence logged to `.omo/evidence/task-16-tofu-core.log`. --- ## Task 13 — `tofu.resolve.generateBuildPlan` (build plan from constrained tree) ### Architecture - New function `generateBuildPlan(constrained, tree, cfg, fetchRecipe = null)` in `tofu.resolve`. - Input: `ConstrainedNode[]` from `constrainDepTree` (already in topological order, deps-first, root last). The `DepTree` parameter is preserved for future context but not used for logic — ordering comes from the constrained array. - Output: `BuildPlan` with `Source.recipe` entries for every recipe-sourced node (binary nodes excluded). - The `fetchRecipe` delegate (`scope string delegate(string) @safe`) is the testability seam. Default `null` means "no fetcher available" — missing recipes throw `FetchException`. - Production wires: `delegate (string name) @safe { return tofu.fetch.fetchRecipe(name, cfg); }`. ### Algorithm 1. Iterate constrained nodes in order (preserves topological ordering). 2. Skip nodes with `source == DepSource.binary` — Zeta handles binary deps. 3. For recipe nodes: compute cache path via `cfg.recipesCacheDir(name) ~ "/" ~ name ~ ".recipe"`. 4. Check `std.file.exists(path)` via `@trusted` wrapper. 5. Missing → if `fetchRecipe` delegate provided, call it; else throw `FetchException("recipe not cached and no fetcher provided")`. 6. Add to `BuildPlan` via `plan.add(name, recipePath, Source.recipe)`. 7. Empty plan → log `"nothing to build (all binary)"`. 8. Non-empty → log `"build plan: N packages"` via `logOk`. ### `@safe` / `@trusted` architecture - `generateBuildPlan` is `@safe`. - Only `std.file.exists` requires `@trusted` wrapper — inline lambda `() @trusted { return exists(path); }()`. - Follows existing patterns from `fetch.d`, `cache.d`, `index.d`. ### FetchException reuse - Uses `tofu.fetch.FetchException` (imported via `import tofu.fetch;`). No new exception class needed — the message `"recipe not cached and no fetcher provided"` is distinct and searchable. ### New imports in resolve.d - `import tofu.config;` — for `Config` type (parameter in function signature). - `import tofu.fetch;` — for `FetchException`. - `import std.file : exists;` — for recipe cache existence check. - `import tofu.log;` already had `logInfo`; now also uses `logStep`, `logOk`. ### Test strategy (6 new unittests, numbered 9–14) - Reused the temp-dir + `scope(exit)` cleanup pattern from other modules. - `version(unittest)` block with `@trusted` helpers: `createCachedRecipe(dir, name)`, `testTempDir(suffix)`, `testRmdir(path)`, `testConfig(cacheDir)`. - Test (9): [C(recipe), B(binary), A(recipe)] → plan [C, A] — B excluded, order preserved. - Test (10): missing cache + fetch delegate → fetch called, returned path used in plan. - Test (11): missing cache + no fetch delegate → FetchException thrown. - Test (12): all binary → empty plan. - Test (13): root included even when dep is binary. - Test (14): recipe files exist in cache → fetch delegate NOT called (delegate throws assert on invocation). ### `scope` on constrained parameter - `scope const ConstrainedNode[] constrained` — DMD 2.112 requires `scope` on array/class reference parameters for `@safe` inference when the function does not escape them. Same pattern as `constrainDepTree` (which uses `scope const PackageIndex[] index`). ### Build verified - `dub test` passes — all 11 modules, including 6 new generateBuildPlan unittests. - `dub build` passes with `warningsAsErrors`. - Pre-existing breakage in `build.d` (parallel task 15 artefact) excluded via `.skip` rename for testing — NOT caused by task 13. - Evidence logged to `.omo/evidence/task-13-tofu-core.log`. --- ## Task 15 — `tofu.build.buildAll` (sequential topological build orchestrator) ### Architecture - `buildAll(BuildPlan plan, Config cfg, bool force = false)` orchestrates sequential package builds with fail-fast semantics. - Added imports: `tofu.types : BuildPlan, BuildResult, BuildFailure, Source;`, `tofu.log : logStep, logOk, logError, logInfo;`. - No new exception classes — reuses existing `BuildException` from `runMakepkg`. ### Algorithm 1. Empty plan → `logInfo("nothing to build")`, return empty `BuildResult`. 2. Iterate `plan.order()` entries (caller guarantees deps-first topological order): a. **Pre-check**: `exists(entry.recipePath)` via `@trusted` wrapper. Missing → `BuildFailure` with "recipe not found at ", return immediately. b. **Separator**: `logStep("──── building %s (%d/%d) ────", name, idx, total)` — U+2500 box drawing chars. c. **Skip-if-exists**: If NOT force AND `cfg.builtDir()/packages//package.lua` exists → logInfo skip, add to succeeded, continue. d. **Execute**: `runMakepkg(entry.recipePath, cfg.builtDir(), cfg.defaultJobs, force, cfg)`. e. **Success**: logOk, add to succeeded. f. **BuildException**: logError, add to failed, return immediately (halt). 3. Return populated `BuildResult`. ### `@safe` / `@trusted` architecture - `buildAll` is `@safe` public. - Only `std.file.exists` requires `@trusted` wrappers — same pattern as other modules. - `log*` functions are `@safe` — no `@trusted` needed for logging. ### Test strategy (6 new unittests, numbered 8–13) - Reused existing test helpers: `testTempDir`, `sWrite`, `sRmdirRecurse`, `makeFakeMakepkg`. - Test (8): plan [depA, depB, target] all valid → all 3 succeed in order, outputs created. - Test (9): fake always exits 1 → only depA fails, succeeded empty, depB/target skipped. - Test (10): empty plan → no-op, both lists empty. - Test (11): pre-existing package.lua + force=false → skipped, fake NOT invoked (args file absent). - Test (12): pre-existing package.lua + force=true → rebuilt, fake invoked (args file present). - Test (13): nonexistent recipe path → fail-fast with "recipe not found", remaining skipped. ### Skip-if-exists semantics - Path checked: `cfg.builtDir() ~ "/packages/" ~ name ~ "/package.lua"` — matches what `runMakepkg` produces (verified at line 190 in `runMakepkg`). - `force=true` bypasses skip — always invokes `runMakepkg`. ### Build verified - `dub test` passes — all 13 modules, including 6 new buildAll unittests. - `dub build` passes with `warningsAsErrors`. - Evidence logged to `.omo/evidence/task-15-tofu-core.log`. --- ## Task 17 — `tofu.install.installAll` (install orchestrator) ### Architecture - `installAll(BuildPlan plan, Config cfg)` added to `tofu.install` — single root-package `-LocalProvide` call. - Imports added: `tofu.types` (selective: `BuildPlan`, `BuildPlanEntry`, `Source`), `std.conv : to`, `std.file : exists`. - Reuses `runLocalProvide` — no new process-spawning logic. ### Algorithm 1. **Empty plan** → `logInfo("nothing to install")`, return early. 2. **Verify built cache**: for each `Source.recipe` entry, check `cfg.builtPackagesDir()/name/package.lua` exists. Missing → `InstallException("built package missing from cache: (was the build skipped?)")`. 3. **Root = last entry** in `plan.order()` — matches `deps.resolve`/`generateBuildPlan` convention (topological order, root last). 4. **Single call**: `runLocalProvide(rootName, cfg)` — ZETA's `deps.resolve` walks the full tree from `ZETA_LOCAL_PACKAGES` + `ZETA_REPO`. 5. **Success**: `logOk("installed with N dependencies")` where N = `order.length - 1`. 6. **InstallException from `runLocalProvide`**: propagates to caller (no catch needed — install command records state). ### "already installed" handling - Handled internally by `runLocalProvide` — exit 0 + "already installed" substring in output → `logInfo("already installed — skipping")`, no throw. - `installAll` continues to `logOk` after. ### Dependencies counted - All entries in `BuildPlan` are recipe-sourced (binary excluded by `generateBuildPlan`). Root is last entry. Dep count = `order.length - 1`. ### `@safe` / `@trusted` architecture - `installAll` is `@safe` public. - Only `std.file.exists` requires `@trusted` wrapper — inline lambda `() @trusted { pkgExists = exists(pkgPath); }()`. ### Imports strategy - Top-level selective import: `import tofu.types : BuildPlan, BuildPlanEntry, Source;` — avoids pulling in the full types module. - `import std.conv : to;` for `to!string(size_t)`. - `import std.file : exists;` at module level for the built-cache verification check. ### Unittests — 5 new test blocks (test 1–5 for installAll) - Reused existing test infrastructure: `makeTempDir`, `removeDir`, `writeFakeScript`, `testConfig`. - Test (1): plan [B, C, A] with all package.lua + fake zeta captures `$2` → "A" only, no "B"/"C". - Test (2): plan missing B's package.lua → `InstallException` "built package missing from cache: B". - Test (3): empty plan → no-op, no throw. - Test (4): fake zeta exits 1 → `InstallException` "install failed for A". - Test (5): fake zeta prints "already installed" + exits 0 → no throw, `logOk` succeeds. ### Arg capture pattern - Fake zeta script: `echo "$2" >> ` — `$2` = package name (args: `fake-zeta -LocalProvide --pass`). - Previous learnings used `$3` incorrectly — in bash, `$0`=script name, `$1`=-LocalProvide, `$2`=pkgName, `$3`=--pass. ### Build verified - `dub test` passes — all 13 modules, including 5 new installAll unittests (plus 5 pre-existing runLocalProvide tests = 10 total in install.d). - `dub build` passes with `warningsAsErrors`. - No D LSP server configured for `.d` — diagnostics verified via compiler. - Evidence logged to `.omo/evidence/task-17-tofu-core.log`. --- ## Task 18 — `tofu.state` (post-install state tracking for -Syu upgrades) ### Architecture - Module `tofu.state` depends on: `tofu.config` (Config, cacheDir), `tofu.log` (logWarn), `std.json` (parseJSON, JSONValue, JSONType), `std.file` (readText, write, rename, exists, remove), `std.datetime` (Clock). - State file: `cfg.cacheDir ~ "/installed.json"` — JSON array of `{"name":"...","ver":"...","installedAt":,"source":"recipe"}` objects. - Five public APIs: `recordInstall`, `listInstalled`, `isInstalledByTofu`, `removeInstallRecord`, `installedVersion`. - Atomic writes: write to `.tmp` file then `rename()` — ensures the state file is never half-written. ### `@safe` / `@trusted` architecture - All public functions are `@safe`. - JSON operations (`parseJSON`, `JSONValue.array` property) are `@system` in Phobos — isolated behind `@trusted` wrappers (`fParseJSON`, `fArray`). - `JSONValue.array` returns `ref inout(JSONValue[])` — must use `ref JSONValue` parameter in the wrapper (not by-value copy) to avoid dangling reference. - Filesystem operations (`readText`, `write`, `rename`, `exists`, `remove`) isolated in `@trusted` wrappers — same pattern as all other modules. ### State file structure - Single JSON array at `~/.cache/tofu/installed.json` — SEPARATE from ZETA's per-package `var/db/zeta//` database. - `recordInstall` handles reinstall: if an entry for `name` already exists, it's replaced (new ver + timestamp); otherwise appended. - `removeInstallRecord` for missing entries is a no-op. - `installedVersion` returns `""` for unknown packages — convenience for upgrade command comparison. - Source is always `"recipe"` — this module only tracks recipe-built packages. ### Corrupted/missing state handling - Missing file → empty list (normal for fresh install). - Corrupted JSON → `logWarn` + return empty list (never throws). - Individual corrupted entries within a valid JSON array → skipped silently. - After corruption, `recordInstall` writes fresh state successfully (recovery). ### Test strategy (7 unittests) - Used temp directory pattern from `config.d`: `tempDir ~ "/tofu-test-state-" ~ suffix ~ "-" ~ thisProcessID.to!string`. - `scope(exit)` with `rmdirRecurse` for cleanup — same as `cache.d`/`build.d`/`install.d`. - `makeTestConfig(suffix)` creates isolated `Config` with unique cache dir per test. - Test (1): recordInstall → listInstalled contains it with correct ver ✓ - Test (2): recordInstall twice same name → single entry, latest ver ✓ - Test (3): isInstalledByTofu → true with filled pkg; unknown → false ✓ - Test (4): removeInstallRecord → gone; removing missing → no-op ✓ - Test (5): corrupted JSON file → empty list + no throw ✓ - Test (6): missing file → empty list ✓ - Test (7): file is valid JSON after writes (parse back externally) ✓ - Extra: installedVersion convenience ✓ ### `@safe` + `@trusted` attribute conflict on test helpers - `makeTestConfig` calls `tempDir` and `thisProcessID` which are `@safe` in DMD 2.112 (not `@system`). Marking it both `@safe` and `@trusted` causes "conflicting attribute" error. Fix: `@safe` only (no `@trusted` suffix needed). ### `JSONType` enum in Phobos - Members: `JSONType.array`, `JSONType.object`, `JSONType.string_` (underscore because `string` is a D keyword), `JSONType.integer`, `JSONType.float_`, etc. - `JSONValue.str` returns `string`, `.integer` returns `long`. ### Build verification - `dub test` (excluding pre-existing broken `cli.d` and `errors.d`) — 14 modules pass unittests including state.d's 7 test blocks. - `dub build` fails due to pre-existing `cli.d` import error (`indexOf` on string) — NOT caused by state.d. state.d compiles clean standalone (`dmd -c -o-`). - Evidence logged to `.omo/evidence/task-18-tofu-core.log`. ### Pre-existing breakage note - `src/tofu/cli.d` and `src/tofu/errors.d` are pre-existing broken modules from parallel agent tasks (after task 17). They block `dub build` and `dub test` (without `.skip` rename). These are outside the scope of task 18. --- ## Task 19 — `tofu.cli` (yay/paru-style command-line parsing) ### Architecture - Module `tofu.cli` — standalone CLI parser, no dependencies on other tofu modules. - Imports: `std.conv` (to, ConvException), `std.string` (startsWith, indexOf). - Manual parsing — no framework dependency. Designed for single-pass argv scanning. ### Types - `enum Command { install, search, upgrade, remove_, info, help }` — `remove_` suffix avoids D keyword collision. - `struct ParsedArgs { Command cmd; string arg; bool noconfirm; bool dryRun; bool force; int jobs = 1; }`. - `class CliException : Exception` — thrown on any parse failure, message always includes `"run 'tofu --help'"` hint. ### Parsing algorithm (single-pass) 1. **Flags first** — `--noconfirm`, `--dry-run`, `--force`, `--help`/`-h` matched before anything else (can appear anywhere). `-h`/`--help` overrides any previously-set command. 2. **-j flag** — two forms: `-j` (attached) or `-j ` (next arg). Validates positive int via `parsePositiveInt()`. Throws `CliException` if missing argument or non-positive. 3. **Command tokens** — only the first one wins (`!cmdSet` guard). Matches exact forms: `-Syu` (upgrade), `-Ss` (search), `-Si` (info), `-S` (install), `-R` (remove). Unknown `-X` tokens before a command → `CliException`. 4. **Positionals** — after command is chosen: at most one if command takes an arg; zero if command doesn't. Extra positionals → `CliException("too many arguments")`. Unexpected args on no-arg commands → `CliException("unexpected argument")`. 5. **Post-scan** — no command → `"no command given"`. Command needs arg but none provided → `"command requires an argument"`. ### `@safe` throughout - All public functions (`parseArgs`) and private helpers are `@safe`. - `parsePositiveInt` uses try-catch on `std.conv.to!int` — `ConvException` caught and re-thrown as `CliException`. - No `@trusted` blocks needed (no filesystem or system calls). ### `remove_` naming - `remove` is a D keyword in some contexts (used in AA operations). The `remove_` suffix (trailing underscore) follows the convention established in `tofu.types` (`BuildResult.failed_`). Callers use `Command.remove_`. ### `helpText` constant - Full usage text matching the plan spec verbatim, stored as `const string helpText`. Named `helpText` (not `usage`) for compatibility with `main.d` which imports `helpText` from `tofu.cli`. ### Test cases (12/12 pass) 1. `-S neovim` → install, arg=neovim ✓ 2. `-Ss editor` → search, arg=editor ✓ 3. `-Syu` → upgrade, arg="" ✓ 4. `-R neovim` → remove ✓ 5. `-Si neovim` → info ✓ 6. `--help` and `-h` → help ✓ 7. empty → CliException "no command given" ✓ 8. `-S neovim --noconfirm --dry-run -j4` → all flags set ✓ 9. `-S neovim -j0` → CliException (must be positive) ✓ 10. unknown flag `-Z` → CliException ✓ 11. `-S neovim extra` → too many arguments ✓ 12. `-Ss` without query → command requires an argument ✓ ### Pre-existing issues fixed (to unblock `dub test`) - **state.d L203**: Conflicting `@safe`/`@trusted` on `makeTestConfig()` — removed `@safe`. - **state.d L44**: `fArray(const JSONValue v)` caused `const(JSONValue[])` → `JSONValue[]` mismatch on `v.array` — removed `const`. - **errors.d L31**: `core.sys.posix.errno` does not exist on DMD 2.112. `ESRCH` lives in `core.stdc.errno` — merged both imports into `import core.stdc.errno : ESRCH, errno;`. - **errors.d L190**: `pid_t` undefined — added `import core.sys.posix.sys.types : pid_t;` in `version(Posix)`. - **errors.d L508**: Missing `indexOf` on string — added `import std.string : indexOf;`. ### `dub build` / `main.d` integration note - `dub test` passes — all 16 modules (including cli.d's 12 unittests) pass. - `dub build` fails because `main.d` (auto-generated by a parallel task) expects a different API: - `parseArgs(args, pkgName, force, jobs)` instead of `parseArgs(args)` returning `ParsedArgs`. - Capitalised enum members (`Command.Help`, `Command.Install`, ...) instead of lowercase. - `signal` handler missing `@nogc` attribute. - This is intentional per the plan — main.d wiring happens in task 20 (the first command implementation task). The cli.d module itself is correct and fully tested. ### Build verified - `dub test` passes — all 16 modules with warnings-as-errors. - Evidence logged to `.omo/evidence/task-19-tofu-core.log`.