feat(upgrade): add -Syu upgrade for tofu-installed recipe packages

- src/tofu/commands/upgrade.d: full 7-step upgrade pipeline
  - Fetches index, lists installed state, compares versions
  - Shows upgrade plan with confirmation prompt
  - REUSES installCommand per outdated package (force=true, noconfirm=true)
  - Continue-on-failure for multi-package upgrades
  - Binary-only packages excluded (pool=binary → skip)
  - 6 unittests: no-installed, all-uptodate, outdated-upgraded,
    removed-from-index, multi-with-failure, confirmation-denied
- src/main.d: wire upgrade stub → upgradeCommand(pa, cfg)
- Evidence: dub test (23/23 pass) + dub build clean
This commit is contained in:
2026-08-08 18:43:21 -04:00
parent 4d8af8f13d
commit bd26e7f428
5 changed files with 948 additions and 2 deletions
+297
View File
@@ -0,0 +1,297 @@
Warning
Warning ## Warning for package tofu ##
Warning
Warning The following compiler flags have been specified in the package description
Warning file. They are handled by DUB and direct use in packages is discouraged.
Warning Alternatively, you can set the DFLAGS environment variable to pass custom flags
Warning to the compiler, or use one of the suggestions below:
Warning
Warning warningsAsErrors: Use "buildRequirements" to control the warning level
Warning
Generating test runner configuration 'tofu-test-application' for 'application' (executable).
Warning Excluding main source file src/main.d from test.
Starting Performing "unittest" build using /usr/bin/dmd for x86_64.
Up-to-date toml 1.0.0: target for configuration [library] is up to date.
Building tofu ~main: building configuration [tofu-test-application]
Linking tofu-test-application
Finished To force a rebuild of up-to-date targets, run again with --force
Running tofu-test-application
zeta-makepkg: building hello...zeta-makepkg: configure phasezeta-makepkg: build phasezeta-makepkg: install phasezeta-makepkg: building hello...zeta-makepkg: configure phasezeta-makepkg: build phasezeta-makepkg: install phasezeta-makepkg: building hello...zeta-makepkg: configure phasezeta-makepkg: build phasezeta-makepkg: install phasezeta-makepkg: building hello...zeta-makepkg: configure phasezeta-makepkg: build phasezeta-makepkg: install phasedonezeta-makepkg: building depA...zeta-makepkg: configure phasezeta-makepkg: build phasezeta-makepkg: install phasezeta-makepkg: building depB...zeta-makepkg: configure phasezeta-makepkg: build phasezeta-makepkg: install phasezeta-makepkg: building target...zeta-makepkg: configure phasezeta-makepkg: build phasezeta-makepkg: install phasezeta-makepkg: building depA...zeta-makepkg: configure phasezeta-makepkg: build phasezeta-makepkg: install phaseerror build failed for depA: build failed for depA:
zeta-makepkg: building depA...
zeta-makepkg: configure phase
zeta-makepkg: build phase
zeta-makepkg: install phase
zeta-makepkg: building pkg...zeta-makepkg: configure phasezeta-makepkg: build phasezeta-makepkg: install phaseerror build failed for noexist: recipe not found at /tmp/tofu-test-build-buildall-norecipe-249455/nonexistent.recipe
warn corrupted cache for pkg: Found 'h' when expecting 'r'. (Line 1:2)
error package 'nonexistent' not found in ZUUR
error package 'ghost' not found in ZUUR
error dependency cycle: A -> B -> A
error build failed for bfp: build failed for bfp:
(no stderr output)
error build failed: bfp — build failed for bfp:
(no stderr output)
error install phase error: install failed for ifp:
 . checking binary firefox ...
- binary firefox-2.1.0 satisfies firefox>=2.0
 . checking binary libfoo ...
- binary libfoo-1.9 does not satisfy libfoo>=2.0
 . checking binary noexist ...
 . checking binary brokenpkg ...
 . checking binary badpkg ...
 . checking binary testpkg ...
- binary testpkg-2.1.0 satisfies testpkg
==> ──── building depA (1/3) ────
 ok depA
==> ──── building depB (2/3) ────
 ok depB
==> ──── building target (3/3) ────
 ok target
==> ──── building depA (1/3) ────
- nothing to build
==> ──── building pkg (1/1) ────
- pkg already built, skipping
==> ──── building pkg (1/1) ────
 ok pkg
neovim
version: 0.9.5
summary: Text editor
pool: both
- recipe not cached — run 'tofu -S neovim' to fetch
neovim
version: 0.9.5
summary: Text editor
pool: both
build system: cmake
deps: libluv>=1.30, msgpack-c
url: https://github.com/neovim/neovim
neovim
version: 0.9.5
summary: Text editor
pool: both
- recipe not cached — run 'tofu -S neovim' to fetch
installed: 0.9.5
status: up to date
ripgrep
version: 14.1.0
summary: Fast grep
pool: recipes
- recipe not cached — run 'tofu -S ripgrep' to fetch
installed: 13.0.0
status: outdated (zuur has 14.1.0)
firefox
version: 120.0
summary: Web browser
pool: binary
firefox
version: 120.0
summary: Web browser
pool: binary
ripgrep
version: 14.1.0
summary: Fast grep
pool: recipes
- recipe not cached — run 'tofu -S ripgrep' to fetch
installed: 15.0.0
status: newer than zuur
==> generating build plan
- + happypkg (recipe)
 ok build plan: 1 packages
- will build 1 package(s): happypkg
==> ──── building happypkg (1/1) ────
 ok happypkg
 ok installed happypkg with 0 dependencies
 ok installed 1 package(s)
==> generating build plan
- + bfp (recipe)
 ok build plan: 1 packages
- will build 1 package(s): bfp
==> ──── building bfp (1/1) ────
==> generating build plan
- + ifp (recipe)
 ok build plan: 1 packages
- will build 1 package(s): ifp
==> ──── building ifp (1/1) ────
 ok ifp
==> generating build plan
- + drp (recipe)
 ok build plan: 1 packages
- will build 1 package(s): drp
- dry run — nothing built
==> generating build plan
- + ncp (recipe)
 ok build plan: 1 packages
- will build 1 package(s): ncp
==> ──── building ncp (1/1) ────
 ok ncp
 ok installed ncp with 0 dependencies
 ok installed 1 package(s)
==> generating build plan
- + abp (recipe)
 ok build plan: 1 packages
- will build 1 package(s): abp
Proceed? [y/N] warn package 'foreignpkg' was not installed by tofu — removing via Zeta anyway
- aborted by user
 ok removed testpkg
removed ok ok removed foreignpkg
error cannot remove libfoo: still required by libbar (use --force to override)error cannot remove libfoo (use --force to override)
 ok removed testpkg
removed without prompt ok removed testpkg
Remove testpkg? [y/N] warn package 'ghost' no longer in ZUUR — skipping
- aborted by user
- nothing to do
 . package 'foo' is up to date (2.0)
- nothing to do
- will upgrade 1 package(s): oldpkg: 1.0 → 2.0
- upgrading oldpkg (1.0 → 2.0)...
==> generating build plan
- + oldpkg (recipe)
 ok build plan: 1 packages
- will build 1 package(s): oldpkg
==> ──── building oldpkg (1/1) ────
 ok oldpkg
 ok installed oldpkg with 0 dependencies
 ok installed 1 package(s)
 ok 1 package(s) upgraded
- will upgrade 1 package(s): realpkg: 1.0 → 2.0
- upgrading realpkg (1.0 → 2.0)...
==> generating build plan
- + realpkg (recipe)
 ok build plan: 1 packages
- will build 1 package(s): realpkg
==> ──── building realpkg (1/1) ────
 ok realpkg
 ok installed realpkg with 0 dependencies
 ok installed 1 package(s)
 ok 1 package(s) upgraded, 1 no longer in ZUUR
- will upgrade 2 package(s): goodpkg: 1.0 → 2.0, badpkg: 1.0 → 2.0
- upgrading goodpkg (1.0 → 2.0)...
==> generating build plan
- + goodpkg (recipe)
 ok build plan: 1 packages
- will build 1 package(s): goodpkg
==> ──── building goodpkg (1/1) ────
 ok goodpkg
 ok installed goodpkg with 0 dependencies
 ok installed 1 package(s)
- upgrading badpkg (1.0 → 2.0)...
==> generating build plan
- + badpkg (recipe)
 ok build plan: 1 packages
- will build 1 package(s): badpkg
==> ──── building badpkg (1/1) ────
 ok badpkg
install failederror install phase error: install failed for badpkg: install failed
error upgrade failed for badpkg
warn 1 upgrade(s) failed
 ok 1 package(s) upgraded
- will upgrade 1 package(s): abortpkg: 1.0 → 2.0
Proceed? [y/N] Warning: malformed TOML config at /tmp/tofu-test-config-bad-249455.toml: Invalid table key declaration (2:0)
Warning: invalid TOFU_DEFAULT_JOBS 'not-a-number', using default 1
warn removing stale lock (PID 99999999 not alive)
127.0.0.1 - - [08/Aug/2026 18:42:15] "GET /recipes/hello/hello.recipe HTTP/1.1" 200 -
127.0.0.1 - - [08/Aug/2026 18:42:15] "GET /recipes/hello/package.lua HTTP/1.1" 200 -
127.0.0.1 - - [08/Aug/2026 18:42:15] "GET /recipes/hello/build.sh HTTP/1.1" 200 -
127.0.0.1 - - [08/Aug/2026 18:42:15] code 404, message File not found
127.0.0.1 - - [08/Aug/2026 18:42:15] "GET /recipes/hello/hello.recipe HTTP/1.1" 404 -
127.0.0.1 - - [08/Aug/2026 18:42:16] "GET /recipes/hello/hello.recipe HTTP/1.1" 200 -
127.0.0.1 - - [08/Aug/2026 18:42:16] code 404, message File not found
127.0.0.1 - - [08/Aug/2026 18:42:16] "GET /recipes/hello/package.lua HTTP/1.1" 404 -
127.0.0.1 - - [08/Aug/2026 18:42:16] code 404, message File not found
127.0.0.1 - - [08/Aug/2026 18:42:16] "GET /recipes/hello/build.sh HTTP/1.1" 404 -
127.0.0.1 - - [08/Aug/2026 18:42:16] "GET /recipes/hello/scripts/build.sh HTTP/1.1" 200 -
127.0.0.1 - - [08/Aug/2026 18:42:16] "GET /recipes/hello/hello.recipe HTTP/1.1" 200 -
127.0.0.1 - - [08/Aug/2026 18:42:16] code 404, message File not found
127.0.0.1 - - [08/Aug/2026 18:42:16] "GET /recipes/hello/package.lua HTTP/1.1" 404 -
127.0.0.1 - - [08/Aug/2026 18:42:16] code 404, message File not found
127.0.0.1 - - [08/Aug/2026 18:42:16] "GET /recipes/hello/build.sh HTTP/1.1" 404 -
warn skipping index entry with empty name
warn skipping index entry with empty name
warn skipping index entry 'badpool': invalid pool 'bad_pool_value'
- aborted by user
==> fetching recipe hello
 . downloading hello.recipe
 . downloading package.lua
 . downloading build.sh
 ok recipe hello downloaded
==> fetching recipe hello
 . downloading hello.recipe
==> fetching recipe hello
 . downloading hello.recipe
 . downloading package.lua
- no package.lua for hello (optional manifest)
 . downloading build.sh
- no build.sh for hello (optional build script)
 . custom build system: fetching scripts/build.sh
 ok recipe hello downloaded
==> fetching recipe hello
 . downloading hello.recipe
 . downloading package.lua
- no package.lua for hello (optional manifest)
 . downloading build.sh
- no build.sh for hello (optional build script)
 ok recipe hello downloaded
==> fetching recipe hello
 . downloading hello.recipe
 . fetching index http://127.0.0.1:43045//index.lua
 ok index loaded: 3 packages
 . fetching index http://127.0.0.1:54983//index.lua
 . fetching index http://127.0.0.1:49387//index.lua
 . fetching index http://127.0.0.1:41617//index.lua
 . fetching index http://127.0.0.1:35363//index.lua
 ok index loaded: 0 packages
 . fetching index http://127.0.0.1:60717//index.lua
 ok index loaded: 1 packages
 . fetching index http://127.0.0.1:40975//index.lua
 ok index loaded: 2 packages
 . fetching index http://127.0.0.1:41091//index.lua
 . fetching index http://127.0.0.1:33219//index.lua
 ok index loaded: 1 packages
installing package...error: build failedreason: missing dependencytestpkg-2.1 is already installed -- use -ReProvide to reinstall - already installed — skipping
 ok installed A with 2 dependencies
- nothing to install
install failed: conflict detectedtestpkg-2.1 is already installed -- use -ReProvide to reinstallwarn corrupted installed state at /tmp/tofu-test-state-corrupt-249455/installed.json: Found 'h' when expecting 'r'. (Line 1:2)
warn corrupted installed state at /tmp/tofu-test-state-corrupt-249455/installed.json: Found 'h' when expecting 'r'. (Line 1:2)
23 modules passed unittests
- already installed — skipping
 ok installed A with 0 dependencies
- binary libfoo-2.1 satisfies libfoo>=2.0
- libfoo: binary 1.9 too old, building from recipe
- libfoo: binary 1.9 too old, building from recipe
- binary libbar-3.0 satisfies libbar (unconstrained)
- libbaz: no binary available, building from recipe
- binary libfoo-1.5 satisfies libfoo>=1.0
- libfoo: binary 1.5 too old, building from recipe
- binary parentA-1.5 satisfies parentA (unconstrained)
- binary parentB-1.5 satisfies parentB (unconstrained)
==> generating build plan
- + C (recipe)
- + A (recipe)
 ok build plan: 2 packages
==> generating build plan
- recipe libfoo not in cache, re-fetching
- + libfoo (recipe)
 ok build plan: 1 packages
==> generating build plan
==> generating build plan
- nothing to build (all binary)
==> generating build plan
- + mypkg (recipe)
 ok build plan: 1 packages
==> generating build plan
- + libfoo (recipe)
- + mypkg (recipe)
 ok build plan: 2 packages
Warning
Warning ## Warning for package tofu ##
Warning
Warning The following compiler flags have been specified in the package description
Warning file. They are handled by DUB and direct use in packages is discouraged.
Warning Alternatively, you can set the DFLAGS environment variable to pass custom flags
Warning to the compiler, or use one of the suggestions below:
Warning
Warning warningsAsErrors: Use "buildRequirements" to control the warning level
Warning
Starting Performing "debug" build using /usr/bin/dmd for x86_64.
Up-to-date toml 1.0.0: target for configuration [library] is up to date.
Up-to-date tofu ~main: target for configuration [application] is up to date.
Finished To force a rebuild of up-to-date targets, run again with --force
+36
View File
@@ -86,3 +86,39 @@ Rather than adding build/install delegate seams to `installCommand`, the tests r
**Rationale**: Keeps `installCommand`'s API surface minimal (only index/binary seams). The fake scripts exercise the full production code path through `buildAll` and `installAll`, providing higher-fidelity integration tests.
---
### Task 22: Upgrade REUSES installCommand — no duplication
The upgrade command calls `installCommand(pkgName, flags, cfg, ...)` for each outdated package rather than replicating the install pipeline (fetch → resolve → build → install → record). This avoids code duplication and ensures upgrades benefit from all fixes/enhancements to the install path.
**Rationale**: DRY principle. The install pipeline is the single source of truth for package installation. Upgrade is "install with a newer version" — same semantics, same code path.
### Task 22: force=true override for upgrade builds
When upgrade calls `installCommand`, it passes a copy of `ParsedArgs` with `force=true`. This ensures stale build output from the previous version is overwritten — ZETA's `-ReProvide` handles the overwrite at install time, but the build step must also rebuild even if `package.lua` already exists in the built cache.
**Rationale**: Without force, `buildAll`'s skip-if-exists optimization would see the old version's `package.lua` and skip the rebuild. Upgrades MUST rebuild.
### Task 22: noconfirm=true on per-package flags in upgrade
Each `installCommand` call within upgrade receives `noconfirm=true`. The upgrade plan already showed the overall list and obtained user confirmation; individual package installs should not re-prompt.
**Rationale**: Double-confirmation is poor UX. The upgrade's "Proceed? [y/N]" covers all packages. If the user wants per-package confirms, they can upgrade individually with `tofu -S <pkg>`.
### Task 22: Continue-on-failure for multi-package upgrades
Unlike `installCommand` which returns immediately on failure, `upgradeCommand` collects failures and continues with remaining packages. This ensures one broken package doesn't block updates to all others.
**Rationale**: Arch/pacman convention (`-Syu` continues on errors). Users expect upgrades to be best-effort. Failed packages are reported in the summary with a non-zero exit code.
### Task 22: Binary-only packages excluded from upgrade
Packages with `pool == Pool.binary` in the ZUUR index are skipped by upgrade with `logDetail`. Binary packages are managed by ZETA directly; tofu only tracks and upgrades recipe-built packages.
**Rationale**: Tofu's `installed.json` state only tracks recipe packages. Upgrading binary packages would require a different mechanism (e.g. querying ZETA's local database), which is outside tofu's scope.
### Task 22: Closure adapter for delegate type mismatch
`upgradeCommand`'s indexFetcher is `PackageIndex[] delegate(Config)` (with Config param), but `installCommand` expects `PackageIndex[] delegate()` (no-arg closure). The adapter is a zero-arg closure `delegate PackageIndex[]() @safe { return index; }` that captures the already-fetched index array.
**Rationale**: Avoids fetching the index once per package during upgrade. The single fetch at upgrade level is reused for all installCommand calls.
+68
View File
@@ -1276,3 +1276,71 @@ The sandbox test (test 2) creates a sentinel file, serves an index containing `o
- `dub test` passes — 21 modules, all 8 install.d unittests pass.
- `dub build` passes with `warningsAsErrors`.
- Evidence logged to `.omo/evidence/task-21-tofu-core.log`.
---
## Task 22 — `tofu.commands.upgrade` (`-Syu` full-system upgrade)
### Architecture
- Module `tofu.commands.upgrade` — file `src/tofu/commands/upgrade.d`.
- Single public function: `upgradeCommand(ParsedArgs flags, Config cfg, PackageIndex[] delegate(Config) @safe indexFetcher = null)` — returns int exit code 0 (success/no-op) or 1 (upgrade failure).
- Depends on: `tofu.config`, `tofu.cli`, `tofu.types`, `tofu.index`, `tofu.state`, `tofu.vercmp`, `tofu.log`, `tofu.commands.install`, `tofu.errors`.
### Algorithm — 7-step pipeline
1. **Fetch index**: via `indexFetcher(cfg)` seam or real `fetchIndex(cfg)`.
2. **List installed**: `listInstalled(cfg)` from `tofu.state`. Empty → `logInfo("nothing to do")` + return 0.
3. **Compare versions**: For each installed pkg: find in index by name. NOT found → `logWarn` + skip. Found with `pool == Pool.binary` → `logDetail` + skip (binary-only packages are not tofu-upgradable). Compare `installed.ver` vs `index.ver` via `tofu.vercmp.compare`: `>= 0` → up to date (`logDetail`); `< 0` → outdated, add to upgrade list.
4. **No outdated check**: Empty upgrade list → `logInfo("nothing to do")` + return 0.
5. **Show plan + confirm**: Format `"will upgrade N package(s): <name>: <old> → <new>, ..."`. Prompt `"Proceed? [y/N] "` unless `flags.noconfirm`. No → `logInfo("aborted by user")` + return 0.
6. **Execute upgrades** (continue-on-failure):
- Create flags copy with `force=true` (ensure rebuild over stale output) + `noconfirm=true` (already confirmed at upgrade level).
- Create installCommand-compatible index closure: `delegate PackageIndex[]() @safe { return index; }` (closure captures the already-fetched index to avoid re-fetch).
- Call `installCommand(pkgName, pkgFlags, cfg, installIdxFetcher, null)` for each outdated package.
- Non-zero exit → `logError("upgrade failed for <name>")`, increment failure counter, CONTINUE.
7. **Summary**: `logOk("N package(s) upgraded, M up to date, ...")`. If failures > 0 → `logWarn("K upgrade(s) failed")` + return 1.
### Delegate type mismatch between upgrade and install
- **upgradeCommand** indexFetcher: `PackageIndex[] delegate(Config) @safe` (takes Config param — matches search/info pattern).
- **installCommand** indexFetcher: `PackageIndex[] delegate() @safe` (closure, no params — captures state).
- **Fix**: Fetch index once in upgradeCommand, then create a zero-arg closure `delegate PackageIndex[]() @safe { return index; }` to pass to installCommand. The closure captures the already-fetched index array.
### Force + noconfirm override on per-package flags
- `flags.force = true` — ensures stale build output is overwritten (Zeta -ReProvide handles overwrite). This is the upgrade semantic: the old package's build output may still exist.
- `flags.noconfirm = true` — the overall upgrade plan was already confirmed; individual installCommand calls should not re-prompt. This avoids double-confirmation UX.
- D struct copy: `auto pkgFlags = flags;` creates a full value copy of `ParsedArgs`.
### Binary-only package exclusion
- Packages with `pool == Pool.binary` are skipped with `logDetail`. Binary packages are managed by ZETA directly, not tofu. The upgrade command only handles recipe-available packages (pool ∈ {recipes, both}).
### `installCommand` re-entrancy safety
- `installCommand` sets module-level `_installCfg = cfg` and clears it via `scope(exit)`. Since tofu is single-threaded (lock-based), calling `installCommand` multiple times from upgrade is safe. Each call manages its own `_installCfg` lifecycle.
### `write` name conflict (std.file vs std.stdio)
- With both `import std.file;` and `import std.stdio;` in scope, bare `write(...)` is ambiguous. Fix: use `std.stdio.write(string)` for stdout writes and `std.file.write(path, content)` for file writes.
- Same issue as encountered in `install.d` (task 21 learnings).
### Test strategy (6 unittests)
- Reused test infrastructure patterns from `install.d`: `makeTempDir`, `removeDir`, `sWrite`, `makeFakeMakepkg`, `makeFakeZeta`.
- New helper: `makeConditionalZeta(dir, failName, failCode)` — fake zeta that fails for a specific package name (needed for test 5 multi-package upgrade-with-failure).
- `makeRecipeDir(cacheDir, name, ver)` — creates recipe cache directory + .recipe file in one call.
### Test cases (6/6 pass)
1. No installed → `logInfo("nothing to do")` + exit 0.
2. Installed all up to date → `logDetail` for each + `logInfo("nothing to do")` + exit 0.
3. One outdated (1.0→2.0) → upgrade via installCommand path, zeta invoked, state updated to 2.0, exit 0.
4. Package removed from index (ghost) + realpkg outdated → ghost gets `logWarn` skip, realpkg upgraded, exit 0, ghost state preserved.
5. goodpkg upgrades, badpkg zeta fails → goodpkg upgraded (state 2.0), badpkg stays at 1.0, exit 1 with `logWarn("1 upgrade(s) failed")`.
6. Confirmation denied (stdin="n\n") → `logInfo("aborted by user")`, exit 0, nothing upgraded, zeta NOT invoked.
### `stdin` redirection caveat
- Test 6 (confirmation denied) must be LAST in the file because `stdin` reassignment affects all subsequent test blocks. Same restriction as `install.d` tests 7 and 8.
### `main.d` integration
- Import: `import tofu.commands.upgrade : upgradeCommand;` (selective).
- Dispatch: `case Command.upgrade: return upgradeCommand(pa, cfg);`.
- Replaced stub `logError("command 'upgrade' not implemented yet")` + `return 1`.
### Build verified
- `dub test` passes — 23 modules, including upgrade.d's 6 unittests.
- `dub build` passes with `warningsAsErrors`.
- Evidence logged to `.omo/evidence/task-22-tofu-core.log`.