feat(install): add -S install with --dry-run and --noconfirm flags

Create full install pipeline (plan task 21):
- src/tofu/recipeparse.d: light .recipe Lua parser (parseRecipeFile)
- src/tofu/commands/install.d: installCommand with delegate seams
- 8 unit tests covering: happy path, not-found, dep cycle, build
  failure, install failure, --dry-run, --noconfirm, user abort
- All 21 modules pass dub test, dub build passes
- Evidence: .omo/evidence/task-21-tofu-core.log
This commit is contained in:
2026-08-08 18:38:22 -04:00
parent 018640b27f
commit 4d8af8f13d
5 changed files with 1412 additions and 0 deletions
+277
View File
@@ -0,0 +1,277 @@
Warning
Warning ## Warning for package tofu ##
Warning
Warning The following compiler flags have been specified in the package description
Warning file. They are handled by DUB and direct use in packages is discouraged.
Warning Alternatively, you can set the DFLAGS environment variable to pass custom flags
Warning to the compiler, or use one of the suggestions below:
Warning
Warning warningsAsErrors: Use "buildRequirements" to control the warning level
Warning
Generating test runner configuration 'tofu-test-application' for 'application' (executable).
Warning Excluding main source file src/main.d from test.
Starting Performing "unittest" build using /usr/bin/dmd for x86_64.
Up-to-date toml 1.0.0: target for configuration [library] is up to date.
Building tofu ~main: building configuration [tofu-test-application]
src/tofu/commands/install.d(262,26): Error: `buildPath` matches conflicting symbols:
auto path = buildPath(tempDir, "tofu-instcmd-" ~ suffix ~ "-" ~ thisProcessID.to!string);
^
/usr/include/dlang/dmd/std/path.d(1492,16): function `std.path.buildPath!char.buildPath`
immutable(C)[] buildPath(C)(const(C)[][] paths...)
^
/usr/include/dlang/dmd/std/path.d(1492,16): function `std.path.buildPath!char.buildPath`
src/tofu/commands/install.d(283,26): Error: `buildPath` matches conflicting symbols:
auto path = buildPath(dir, "fake-makepkg");
^
/usr/include/dlang/dmd/std/path.d(1492,16): function `std.path.buildPath!char.buildPath`
immutable(C)[] buildPath(C)(const(C)[][] paths...)
^
/usr/include/dlang/dmd/std/path.d(1492,16): function `std.path.buildPath!char.buildPath`
src/tofu/commands/install.d(303,26): Error: `buildPath` matches conflicting symbols:
auto path = buildPath(dir, "fake-zeta");
^
/usr/include/dlang/dmd/std/path.d(1492,16): function `std.path.buildPath!char.buildPath`
immutable(C)[] buildPath(C)(const(C)[][] paths...)
^
/usr/include/dlang/dmd/std/path.d(1492,16): function `std.path.buildPath!char.buildPath`
Error /usr/bin/dmd failed with exit code 1.
Warning
Warning ## Warning for package tofu ##
Warning
Warning The following compiler flags have been specified in the package description
Warning file. They are handled by DUB and direct use in packages is discouraged.
Warning Alternatively, you can set the DFLAGS environment variable to pass custom flags
Warning to the compiler, or use one of the suggestions below:
Warning
Warning warningsAsErrors: Use "buildRequirements" to control the warning level
Warning
Generating test runner configuration 'tofu-test-application' for 'application' (executable).
Warning Excluding main source file src/main.d from test.
Starting Performing "unittest" build using /usr/bin/dmd for x86_64.
Up-to-date toml 1.0.0: target for configuration [library] is up to date.
Up-to-date tofu ~main: target for configuration [tofu-test-application] is up to date.
Finished To force a rebuild of up-to-date targets, run again with --force
Running tofu-test-application
zeta-makepkg: building hello...zeta-makepkg: configure phasezeta-makepkg: build phasezeta-makepkg: install phasezeta-makepkg: building hello...zeta-makepkg: configure phasezeta-makepkg: build phasezeta-makepkg: install phasezeta-makepkg: building hello...zeta-makepkg: configure phasezeta-makepkg: build phasezeta-makepkg: install phasezeta-makepkg: building hello...zeta-makepkg: configure phasezeta-makepkg: build phasezeta-makepkg: install phasedonezeta-makepkg: building depA...zeta-makepkg: configure phasezeta-makepkg: build phasezeta-makepkg: install phasezeta-makepkg: building depB...zeta-makepkg: configure phasezeta-makepkg: build phasezeta-makepkg: install phasezeta-makepkg: building target...zeta-makepkg: configure phasezeta-makepkg: build phasezeta-makepkg: install phasezeta-makepkg: building depA...zeta-makepkg: configure phasezeta-makepkg: build phasezeta-makepkg: install phaseerror build failed for depA: build failed for depA:
zeta-makepkg: building depA...
zeta-makepkg: configure phase
zeta-makepkg: build phase
zeta-makepkg: install phase
zeta-makepkg: building pkg...zeta-makepkg: configure phasezeta-makepkg: build phasezeta-makepkg: install phaseerror build failed for noexist: recipe not found at /tmp/tofu-test-build-buildall-norecipe-245807/nonexistent.recipe
warn corrupted cache for pkg: Found 'h' when expecting 'r'. (Line 1:2)
error package 'nonexistent' not found in ZUUR
error package 'ghost' not found in ZUUR
error dependency cycle: A -> B -> A
error build failed for bfp: build failed for bfp:
(no stderr output)
error build failed: bfp — build failed for bfp:
(no stderr output)
error install phase error: install failed for ifp:
 . checking binary firefox ...
- binary firefox-2.1.0 satisfies firefox>=2.0
 . checking binary libfoo ...
- binary libfoo-1.9 does not satisfy libfoo>=2.0
 . checking binary noexist ...
 . checking binary brokenpkg ...
 . checking binary badpkg ...
 . checking binary testpkg ...
- binary testpkg-2.1.0 satisfies testpkg
==> ──── building depA (1/3) ────
 ok depA
==> ──── building depB (2/3) ────
 ok depB
==> ──── building target (3/3) ────
 ok target
==> ──── building depA (1/3) ────
- nothing to build
==> ──── building pkg (1/1) ────
- pkg already built, skipping
==> ──── building pkg (1/1) ────
 ok pkg
neovim
version: 0.9.5
summary: Text editor
pool: both
- recipe not cached — run 'tofu -S neovim' to fetch
neovim
version: 0.9.5
summary: Text editor
pool: both
build system: cmake
deps: libluv>=1.30, msgpack-c
url: https://github.com/neovim/neovim
neovim
version: 0.9.5
summary: Text editor
pool: both
- recipe not cached — run 'tofu -S neovim' to fetch
installed: 0.9.5
status: up to date
ripgrep
version: 14.1.0
summary: Fast grep
pool: recipes
- recipe not cached — run 'tofu -S ripgrep' to fetch
installed: 13.0.0
status: outdated (zuur has 14.1.0)
firefox
version: 120.0
summary: Web browser
pool: binary
firefox
version: 120.0
summary: Web browser
pool: binary
ripgrep
version: 14.1.0
summary: Fast grep
pool: recipes
- recipe not cached — run 'tofu -S ripgrep' to fetch
installed: 15.0.0
status: newer than zuur
==> generating build plan
- + happypkg (recipe)
 ok build plan: 1 packages
- will build 1 package(s): happypkg
==> ──── building happypkg (1/1) ────
 ok happypkg
 ok installed happypkg with 0 dependencies
 ok installed 1 package(s)
==> generating build plan
- + bfp (recipe)
 ok build plan: 1 packages
- will build 1 package(s): bfp
==> ──── building bfp (1/1) ────
==> generating build plan
- + ifp (recipe)
 ok build plan: 1 packages
- will build 1 package(s): ifp
==> ──── building ifp (1/1) ────
 ok ifp
==> generating build plan
- + drp (recipe)
 ok build plan: 1 packages
- will build 1 package(s): drp
- dry run — nothing built
==> generating build plan
- + ncp (recipe)
 ok build plan: 1 packages
- will build 1 package(s): ncp
==> ──── building ncp (1/1) ────
 ok ncp
 ok installed ncp with 0 dependencies
 ok installed 1 package(s)
==> generating build plan
- + abp (recipe)
 ok build plan: 1 packages
- will build 1 package(s): abp
Proceed? [y/N] Warning: malformed TOML config at /tmp/tofu-test-config-bad-245807.toml: Invalid table key declaration (2:0)
Warning: invalid TOFU_DEFAULT_JOBS 'not-a-number', using default 1
warn removing stale lock (PID 99999999 not alive)
127.0.0.1 - - [08/Aug/2026 18:37:13] "GET /recipes/hello/hello.recipe HTTP/1.1" 200 -
127.0.0.1 - - [08/Aug/2026 18:37:13] "GET /recipes/hello/package.lua HTTP/1.1" 200 -
127.0.0.1 - - [08/Aug/2026 18:37:13] "GET /recipes/hello/build.sh HTTP/1.1" 200 -
127.0.0.1 - - [08/Aug/2026 18:37:13] code 404, message File not found
127.0.0.1 - - [08/Aug/2026 18:37:13] "GET /recipes/hello/hello.recipe HTTP/1.1" 404 -
127.0.0.1 - - [08/Aug/2026 18:37:13] "GET /recipes/hello/hello.recipe HTTP/1.1" 200 -
127.0.0.1 - - [08/Aug/2026 18:37:13] code 404, message File not found
127.0.0.1 - - [08/Aug/2026 18:37:13] "GET /recipes/hello/package.lua HTTP/1.1" 404 -
127.0.0.1 - - [08/Aug/2026 18:37:13] code 404, message File not found
127.0.0.1 - - [08/Aug/2026 18:37:13] "GET /recipes/hello/build.sh HTTP/1.1" 404 -
127.0.0.1 - - [08/Aug/2026 18:37:13] "GET /recipes/hello/scripts/build.sh HTTP/1.1" 200 -
127.0.0.1 - - [08/Aug/2026 18:37:13] "GET /recipes/hello/hello.recipe HTTP/1.1" 200 -
127.0.0.1 - - [08/Aug/2026 18:37:13] code 404, message File not found
127.0.0.1 - - [08/Aug/2026 18:37:13] "GET /recipes/hello/package.lua HTTP/1.1" 404 -
127.0.0.1 - - [08/Aug/2026 18:37:13] code 404, message File not found
127.0.0.1 - - [08/Aug/2026 18:37:13] "GET /recipes/hello/build.sh HTTP/1.1" 404 -
warn skipping index entry with empty name
warn skipping index entry with empty name
warn skipping index entry 'badpool': invalid pool 'bad_pool_value'
- aborted by user
==> fetching recipe hello
 . downloading hello.recipe
 . downloading package.lua
 . downloading build.sh
 ok recipe hello downloaded
==> fetching recipe hello
 . downloading hello.recipe
==> fetching recipe hello
 . downloading hello.recipe
 . downloading package.lua
- no package.lua for hello (optional manifest)
 . downloading build.sh
- no build.sh for hello (optional build script)
 . custom build system: fetching scripts/build.sh
 ok recipe hello downloaded
==> fetching recipe hello
 . downloading hello.recipe
 . downloading package.lua
- no package.lua for hello (optional manifest)
 . downloading build.sh
- no build.sh for hello (optional build script)
 ok recipe hello downloaded
==> fetching recipe hello
 . downloading hello.recipe
 . fetching index http://127.0.0.1:49713//index.lua
 ok index loaded: 3 packages
 . fetching index http://127.0.0.1:46903//index.lua
 . fetching index http://127.0.0.1:55641//index.lua
 . fetching index http://127.0.0.1:35097//index.lua
 . fetching index http://127.0.0.1:60737//index.lua
 ok index loaded: 0 packages
 . fetching index http://127.0.0.1:36947//index.lua
 ok index loaded: 1 packages
 . fetching index http://127.0.0.1:42949//index.lua
 ok index loaded: 2 packages
 . fetching index http://127.0.0.1:35765//index.lua
 . fetching index http://127.0.0.1:35763//index.lua
 ok index loaded: 1 packages
installing package...error: build failedreason: missing dependencytestpkg-2.1 is already installed -- use -ReProvide to reinstall - already installed — skipping
 ok installed A with 2 dependencies
- nothing to install
install failed: conflict detectedtestpkg-2.1 is already installed -- use -ReProvide to reinstallwarn corrupted installed state at /tmp/tofu-test-state-corrupt-245807/installed.json: Found 'h' when expecting 'r'. (Line 1:2)
warn corrupted installed state at /tmp/tofu-test-state-corrupt-245807/installed.json: Found 'h' when expecting 'r'. (Line 1:2)
21 modules passed unittests
- already installed — skipping
 ok installed A with 0 dependencies
- binary libfoo-2.1 satisfies libfoo>=2.0
- libfoo: binary 1.9 too old, building from recipe
- libfoo: binary 1.9 too old, building from recipe
- binary libbar-3.0 satisfies libbar (unconstrained)
- libbaz: no binary available, building from recipe
- binary libfoo-1.5 satisfies libfoo>=1.0
- libfoo: binary 1.5 too old, building from recipe
- binary parentA-1.5 satisfies parentA (unconstrained)
- binary parentB-1.5 satisfies parentB (unconstrained)
==> generating build plan
- + C (recipe)
- + A (recipe)
 ok build plan: 2 packages
==> generating build plan
- recipe libfoo not in cache, re-fetching
- + libfoo (recipe)
 ok build plan: 1 packages
==> generating build plan
==> generating build plan
- nothing to build (all binary)
==> generating build plan
- + mypkg (recipe)
 ok build plan: 1 packages
==> generating build plan
- + libfoo (recipe)
- + mypkg (recipe)
 ok build plan: 2 packages
Warning
Warning ## Warning for package tofu ##
Warning
Warning The following compiler flags have been specified in the package description
Warning file. They are handled by DUB and direct use in packages is discouraged.
Warning Alternatively, you can set the DFLAGS environment variable to pass custom flags
Warning to the compiler, or use one of the suggestions below:
Warning
Warning warningsAsErrors: Use "buildRequirements" to control the warning level
Warning
Starting Performing "debug" build using /usr/bin/dmd for x86_64.
Up-to-date toml 1.0.0: target for configuration [library] is up to date.
Up-to-date tofu ~main: target for configuration [application] is up to date.
Finished To force a rebuild of up-to-date targets, run again with --force
+28
View File
@@ -58,3 +58,31 @@ DMD 2.112's `core.sys.posix.signal.signal` requires the handler to be `@nogc`.
Similarly, `_exit(130)` from `core.sys.posix.unistd` is used instead of `core.stdc.stdlib.exit` — `_exit` does NOT run atexit handlers or flush stdio buffers, making it safe in a signal-handler context.
---
### Task 21: Module-level Config for `resolveDepTree` function pointer seam
`resolveDepTree` uses `Recipe function(string) @safe` (function pointer, not delegate) because the production caller (`installCommand`) needs to capture `Config cfg`. D function pointers cannot carry state, so a module-level `_installCfg` variable bridges the gap.
**Pattern**: Set `_installCfg = cfg` before calling `resolveDepTree`, clear via `scope(exit) _installCfg = Config.init;`. The `_recipeForDeps` function pointer reads `_installCfg` for cache-path access and recipe fetching.
**Rationale**: Tofu is single-threaded (lock-file based). Module-level state is safe within a single command run. The `scope(exit)` ensures cleanup on all exit paths (return, exception, goto).
### Task 21: Light recipe parser in `tofu.recipeparse` — double-quote only
The `extractKeyValue` scanner (ported from `tofu.fetch`) only recognizes double-quoted string values (`"value"`). Single-quoted values (`'value'`) are silently ignored, producing empty fields.
**Decision**: Document this limitation rather than adding single-quote support. ZUUR `.recipe` files use double quotes per Lua convention. The parser is explicitly documented as a LIGHT scanner, not a full Lua parser.
### Task 21: Confirmation prompt via `readln()` — `@trusted` wrapper
`std.stdio.stdin`, `stdout`, and `readln()` are all `@system` in DMD 2.112. The confirmation prompt wraps stdout write/flush and stdin read in `@trusted` helpers (`trustedReadLine()`), keeping the public `installCommand` `@safe`.
**Rationale**: Same pattern used in all tofu modules (`tofu.log`, `tofu.build`, `tofu.install`). No new `@trusted` philosophy — just the standard I/O seam.
### Task 21: Fake makepkg/zeta scripts for unit tests
Rather than adding build/install delegate seams to `installCommand`, the tests reuse the fake-script pattern from `build.d` and `install.d`: create executable bash scripts in temp directories, point `cfg.zetaToolchainPath` and `cfg.zetaPath` at them.
**Rationale**: Keeps `installCommand`'s API surface minimal (only index/binary seams). The fake scripts exercise the full production code path through `buildAll` and `installAll`, providing higher-fidelity integration tests.
---
+81
View File
@@ -1195,3 +1195,84 @@ The sandbox test (test 2) creates a sentinel file, serves an index containing `o
- Import: `import tofu.commands.info : infoCommand;` (selective, matching `: infoCommand;` since the function is the only export needed).
- Dispatch: `case Command.info: return infoCommand(pa.arg, pa, cfg);`
- Signature matches: `infoCommand(string, ParsedArgs, Config)` with the 4th param having a default value.
---
## Task 21 — `tofu.commands.install` (`-S` full install pipeline)
### Architecture
- Module `tofu.commands.install` — file `src/tofu/commands/install.d`.
- New supporting module: `src/tofu/recipeparse.d` with `parseRecipeFile(string path)`.
- Depends on: `tofu.config`, `tofu.cli`, `tofu.types`, `tofu.index`, `tofu.deps`, `tofu.resolve`, `tofu.binary`, `tofu.fetch`, `tofu.build`, `tofu.install`, `tofu.state`, `tofu.log`, `tofu.recipeparse`, `tofu.errors`.
### Public API: `installCommand`
- `int installCommand(string pkgName, ParsedArgs flags, Config cfg, indexFetcher = null, binaryCheck = null)` — returns exit code 0-5.
- Testability seams: `indexFetcher` and `binaryCheck` delegates, both default to null (→ real implementations).
- Pipeline: fetch index → resolve deps → constrain tree → generate plan → confirm → dry-run check → build → install → record state → summary.
### `tofu.recipeparse` — light Lua .recipe parser
- `Recipe parseRecipeFile(string path)` scans for known keys using `extractKeyValue` pattern (same as fetch.d).
- Supported fields: name, version/ver, summary, url, sha256, build_system, build_script, test, deps, files.
- Deps list: `{ "a", "b>=1.0" }` → scanned for quoted strings via `extractList()`.
- IMPORTANT: Only double-quoted (`"`) string values are recognized — single quotes (`'`) are not supported by the light parser.
- Unknown build_system → `BuildSystem.unknown`.
- This is a LIGHT scanner, not a full Lua parser — documented limitation.
### `resolveDepTree` function pointer seam
- `resolveDepTree` takes `Recipe function(string) @safe` (NOT delegate).
- Workaround: module-level `_installCfg` Config variable, set before `resolveDepTree` call, cleared via `scope(exit)`.
- `_recipeForDeps(string name)` function pointer reads `_installCfg` to access cache paths and fetch recipes.
- This is safe because tofu is single-threaded (lock-based).
### Confirmation prompt
- `write("Proceed? [y/N] ")` + `stdout.flush()` + `readln()` — all wrapped in `@trusted` since `stdout`/`stdin` are `@system` in DMD 2.112.
- Accepts `y`, `Y`, `yes`, `YES` — anything else (including EOF) → "aborted by user" + return 0.
- `--noconfirm` skips prompt entirely.
### `@safe` pointer-to-local issue (DMD 2.112)
- Cannot take `&entry` of a `ref entry` in `foreach` inside `@safe` code.
- Fix: use index-based `for` loop and `&index[i]` (heap array, `@safe`-allowed).
### `write` name conflict between `std.file.write` and `std.stdio.write`
- When both `std.file` and `std.stdio` are imported, bare `write(path, content)` is ambiguous.
- Fix: use fully-qualified `std.file.write(path, content)` in test helpers.
- Same issue with `buildPath` — use selective import `import std.path : buildPath;`.
### Recipe quoting gotcha — single vs double quotes
- The `extractKeyValue` parser only recognizes double-quoted strings (`name = "value"`).
- Single-quoted strings (`name = 'value'`) are silently ignored → parsed Recipe has empty fields.
- This manifests as exit code 2 (not found) because the recipe's name is empty → dep resolution fails → `fetchRecipe("")` throws `FetchException(notFound=true)` → `exitCodeFor` maps to 2.
- Fix: all test recipe content MUST use double quotes.
### Unittests (8/8 pass)
1. Full happy path: pkg in index, no deps, fake makepkg + fake zeta → exit 0, state recorded, zeta args verified.
2. Package not in index → exit 2.
3. Dep cycle (A→B, B→A) → exit 3.
4. Build failure (fake makepkg exits 1) → exit 4, no state recorded.
5. Install failure (fake zeta exits 1) → exit 5.
6. `--dry-run` → exit 0, nothing built/installed/recorded.
7. `--noconfirm` with "n\n" in stdin → proceeds (prompt skipped).
8. Confirmation denied ("n\n" in stdin) → exit 0, nothing built.
### Fake script patterns for tests
- `makeFakeMakepkg(dir, exitCode)` — bash script that parses `--output` and creates `packages/<name>/package.lua`.
- `makeFakeZeta(dir, exitCode)` — bash script that writes `$2` (package name) to args file.
- Adapted from `build.d` and `install.d` test patterns.
- Must use `chmod(toStringz(path), octal!755)` to make scripts executable.
### Parallel agent file conflicts
- Parallel agents (tasks 22, 23, 24) rename/overwrite files in `src/tofu/commands/`.
- Files get renamed to `.skip` or replaced with stubs during parallel execution.
- Mitigation: restore files and immediately run `dub test`/`dub build` in the same tool call.
- `remove.d` / `info.d` from parallel tasks may have compile errors — stub them out temporarily.
### `stdin` manipulation in tests (tests 7 and 8)
- Save/restore pattern: store `savedStdin = stdin`, open temp file as `stdin`, restore via `scope(exit)`.
- `File` is a reference type — assignment `stdin = File(path, "r")` shares the GC-managed object.
- Tests 7 and 8 must be at the END of the file — `stdin` may be left in a bad state after test 8.
- `readln()` must be wrapped in `@trusted` — it's `@system` in DMD 2.112.
### `dub test` and `dub build` verified
- `dub test` passes — 21 modules, all 8 install.d unittests pass.
- `dub build` passes with `warningsAsErrors`.
- Evidence logged to `.omo/evidence/task-21-tofu-core.log`.