huntedbytheirs/tofu
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(binary): check zuur/binary package.lua version against constraints

Browse Source
This commit is contained in:
huntedbytheirs
2026-08-08 17:38:21 -04:00
parent c876821cda
commit b367f21d29
3 changed files with 469 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.omo/evidence/task-10-tofu-core.log
+27
View File
@@ -0,0 +1,27 @@
Warning
Warning ## Warning for package tofu ##
Warning
Warning The following compiler flags have been specified in the package description
Warning file. They are handled by DUB and direct use in packages is discouraged.
Warning Alternatively, you can set the DFLAGS environment variable to pass custom flags
Warning to the compiler, or use one of the suggestions below:
Warning
Warning warningsAsErrors: Use "buildRequirements" to control the warning level
Warning
Starting Performing "debug" build using /usr/bin/dmd for x86_64.
Up-to-date toml 1.0.0: target for configuration [library] is up to date.
Building tofu ~main: building configuration [application]
Linking tofu
Finished To force a rebuild of up-to-date targets, run again with --force
Warning: malformed TOML config at /tmp/tofu-test-config-bad-190867.toml: Invalid table key declaration (2:0)
Warning: invalid TOFU_DEFAULT_JOBS 'not-a-number', using default 1
7 modules passed unittests
 . checking binary firefox ...
- binary firefox-2.1.0 satisfies firefox>=2.0
 . checking binary libfoo ...
- binary libfoo-1.9 does not satisfy libfoo>=2.0
 . checking binary noexist ...
 . checking binary brokenpkg ...
 . checking binary badpkg ...
 . checking binary testpkg ...
- binary testpkg-2.1.0 satisfies testpkg
.omo/notepads/tofu-core/learnings.md
+52
View File
@@ -183,6 +183,58 @@ _Auto-scaffolded by /start-work. Append new entries below - never overwrite._
---
## Task 10 — `tofu.binary` (zuur/binary package.lua version checking)
### Architecture
- Module `tofu.binary` depends on: `tofu.types` (BinaryCheckResult, DepConstraint), `tofu.http` (get, HttpException), `tofu.config` (Config, binaryManifestUrl), `tofu.vercmp` (satisfies), `tofu.log` (logDetail, logInfo).
- `checkBinaryVersion(name, constraint, cfg)` fetches the package.lua manifest from zuur/binary via HTTP, parses the version, and checks satisfaction.
### Key logic — 404 vs other errors
- HTTP 404 → binary doesn't exist (recipe-only). Return `BinaryCheckResult(false, "", false)`.
- Any other HttpException (500, timeout, connection failure) → **rethrow** — network problems must propagate, not be silently swallowed.
- Detection: `e.msg.canFind("404")` — the HttpException format is `"HTTP %d fetching <url>"`.
### Version extraction from Lua package.lua
- Light parse: scan for `version = "..."` in the manifest body.
- Word-boundary check on "version" keyword (preceding char must be whitespace/`{`/`,`/`;`, following char must be whitespace/`=`).
- If version field not found or malformed → `BinaryException`.
- This avoids a full Lua parser — package.lua manifests are flat key-value tables.
- Empty version string (e.g. `version = ""`) also throws BinaryException.
### Memoization
- Module-level `BinaryCheckResult[string] _versionCache` — keyed by package name.
- On cache hit: recompute satisfaction against caller's constraint (same version, different constraint possible).
- On 404: cache the "not found" result so subsequent calls don't re-fetch.
- Lifetime: one command run. Tests use distinct package names to avoid cross-state contamination.
- The cache stores `exists` and `ver`; `satisfies` is always recomputed per-constraint.
### `version` keyword trap (again)
- Local variable `version` conflicts with D keyword. Used `ver` instead. Same pattern as types.d where struct field is `ver`.
### Test infrastructure
- Reused local one-shot TCP server pattern from http.d:
- `bindAndSpawn(response)` — bind ephemeral port, spawn thread, return URL.
- `oneShotResponder(listener, response)` — accept one connection, send canned HTTP response, exit.
- `httpResponse(code, reason, body)` — build minimal HTTP/1.1 response with Content-Length.
- `testConfig(baseUrl)` — create a Config pointing zuurUrl at the test server.
### Test cases (6/6 pass)
1. Binary exists, version 2.1.0, constraint ge 2.0 → satisfies:true ✓
2. Binary exists, version 1.9, constraint ge 2.0 → satisfies:false ✓
3. 404 → exists:false, ver:"", satisfies:false ✓
4. 500 → HttpException rethrown (not swallowed) ✓
5. Manifest without version field → BinaryException ✓
6. Version 2.1.0 with unconstrained dep (op none) → satisfies:true ✓
### `@safe` annotation consistency
- `checkBinaryVersion` and all helpers marked `@safe`. The function calls `http.get()` (which is `@safe`), `satisfies` (`@safe`), and log functions (`@safe`). No `@trusted` blocks needed in production code.
- Test helpers (`oneShotResponder`, `bindAndSpawn`, `httpResponse`) are `@trusted` since they use `std.socket`.
### Note: pre-existing fetch.d compile issue
- `fetch.d` (from parallel task 7) has a missing `std.algorithm.searching : canFind` import that blocks `dub test`. `dub build` passes (fetch.d is excluded via `.skip` extension renames by the parallel agent). Binary.d tests verified via standalone `dmd -I... -i -main -unittest` compilation — all 7 modules pass.
---
## Task 7 — `tofu.index` (ZUUR index fetch & sandboxed Lua parse)
### Security: whitelist sandbox (NOT blacklist)
src/tofu/binary.d
+390
View File
@@ -0,0 +1,390 @@
/// tofu.binary — Check zuur/binary package.lua versions against dependency constraints.
///
/// Fetches the package.lua manifest from the zuur binary repository via tofu.http,
/// extracts the version field with a light-parse scan, and checks it against a
/// dependency constraint using tofu.vercmp.satisfies.
///
/// Memoization: Results are cached per package name at module level for the
/// lifetime of one command run. Tests must not rely on cross-test state;
/// each test case uses a fresh or distinct package name.
module tofu.binary;
import tofu.types; // BinaryCheckResult, DepConstraint, DepOp
import tofu.http; // get, HttpException
import tofu.config; // Config, binaryManifestUrl
import tofu.vercmp; // satisfies
import tofu.log; // logDetail, logInfo
import std.algorithm.searching : canFind;
// ────────────────────────────────────────────────────────────
// Exception
// ────────────────────────────────────────────────────────────
/// Thrown when the version field cannot be determined from a
/// binary package's manifest.
class BinaryException : Exception
{
@safe this(string msg)
{
super(msg);
}
}
// ────────────────────────────────────────────────────────────
// Module-level memoization cache
// ────────────────────────────────────────────────────────────
/// Cache of manifest fetch results keyed by package name.
/// Shared across all calls within a single command run.
/// Tests must NOT rely on cross-test state.
private BinaryCheckResult[string] _versionCache;
// ────────────────────────────────────────────────────────────
// Helpers
// ────────────────────────────────────────────────────────────
/// Light-parse the version field from a Lua package.lua manifest.
/// Uses a simple scan for `version = "..."` — package.lua manifests
/// are flat key-value tables; a full Lua parser is overkill for
/// version extraction.
private @safe string extractVersion(string content, string name)
{
size_t idx = 0;
// Find "version" keyword with word-boundary checks
while (idx + 7 <= content.length)
{
if (content[idx .. idx + 7] == "version")
{
// Validate preceding char (word boundary)
bool validBefore = (idx == 0);
if (!validBefore)
{
char c = content[idx - 1];
validBefore = (c == ' ' || c == '\t' || c == '\n' || c == '\r'
|| c == '{' || c == ',' || c == ';');
}
// Validate following char (word boundary)
bool validAfter = (idx + 7 >= content.length);
if (!validAfter)
{
char c = content[idx + 7];
validAfter = (c == ' ' || c == '\t' || c == '=');
}
if (validBefore && validAfter)
break;
}
idx++;
}
if (idx + 7 > content.length)
{
throw new BinaryException(
"cannot determine version for binary package '" ~ name ~ "'");
}
idx += 7; // skip "version"
// Skip whitespace before '='
while (idx < content.length && (content[idx] == ' ' || content[idx] == '\t'))
idx++;
// Expect '='
if (idx >= content.length || content[idx] != '=')
{
throw new BinaryException(
"cannot determine version for binary package '" ~ name ~ "'");
}
idx++;
// Skip whitespace after '='
while (idx < content.length && (content[idx] == ' ' || content[idx] == '\t'))
idx++;
// Expect opening quote
if (idx >= content.length || content[idx] != '"')
{
throw new BinaryException(
"cannot determine version for binary package '" ~ name ~ "'");
}
idx++;
// Read version value until closing quote
size_t verStart = idx;
while (idx < content.length && content[idx] != '"')
idx++;
if (idx >= content.length)
{
throw new BinaryException(
"cannot determine version for binary package '" ~ name ~ "'");
}
auto ver = content[verStart .. idx];
if (ver.length == 0)
{
throw new BinaryException(
"cannot determine version for binary package '" ~ name ~ "'");
}
return ver;
}
/// Format a dep constraint for human-readable log messages.
private @safe string constraintToString(DepConstraint c)
{
final switch (c.op)
{
case DepOp.none:
return c.name;
case DepOp.ge:
return c.name ~ ">=" ~ c.ver;
case DepOp.le:
return c.name ~ "<=" ~ c.ver;
case DepOp.eq:
return c.name ~ "==" ~ c.ver;
case DepOp.ne:
return c.name ~ "~=" ~ c.ver;
case DepOp.gt:
return c.name ~ ">" ~ c.ver;
case DepOp.lt:
return c.name ~ "<" ~ c.ver;
}
}
// ────────────────────────────────────────────────────────────
// Public API
// ────────────────────────────────────────────────────────────
/// Check a binary package's version against a dependency constraint.
///
/// Flow:
/// 1. Check module-level cache for previously fetched manifest.
/// 2. Fetch `cfg.binaryManifestUrl(name)` via `tofu.http.get`.
/// 3. On HTTP 404 → return BinaryCheckResult(exists:false) —
/// treat as recipe-only, no binary available.
/// 4. On any other HttpException → rethrow (network/timeout errors
/// must propagate, not be silently swallowed).
/// 5. Parse the package.lua to extract `version` via light regex scan.
/// If version not found → throw BinaryException.
/// 6. Check `tofu.vercmp.satisfies(version, constraint)`.
/// 7. Cache raw result (exists + ver) for subsequent calls; satisfaction
/// is recomputed against each caller's constraint on cache hit.
///
/// Returns:
/// BinaryCheckResult with exists, version string, and satisfaction flag.
@safe BinaryCheckResult checkBinaryVersion(string name, DepConstraint constraint,
Config cfg)
{
// 1. Check cache — recompute satisfaction against caller's constraint
if (auto cached = name in _versionCache)
{
bool satisfied = cached.exists && satisfies(cached.ver, constraint);
return BinaryCheckResult(cached.exists, cached.ver, satisfied);
}
logDetail("checking binary %s ...", name);
// 2. Fetch the package.lua manifest
string manifestContent;
try
{
manifestContent = get(cfg.binaryManifestUrl(name));
}
catch (HttpException e)
{
// 3. HTTP 404 → binary doesn't exist (recipe-only)
if (e.msg.canFind("404"))
{
auto result = BinaryCheckResult(false, "", false);
_versionCache[name] = result;
return result;
}
// 4. Any other HTTP error → rethrow (network problems must propagate)
throw e;
}
// 5. Extract version from manifest (local name 'ver' — 'version' is a D keyword)
auto ver = extractVersion(manifestContent, name);
// 6. Check satisfaction
bool satisfied = satisfies(ver, constraint);
// 7. Log result
if (satisfied)
logInfo("binary %s-%s satisfies %s", name, ver, constraintToString(constraint));
else
logInfo("binary %s-%s does not satisfy %s", name, ver, constraintToString(constraint));
// 8. Cache and return
auto result = BinaryCheckResult(true, ver, satisfied);
_versionCache[name] = result;
return result;
}
// ────────────────────────────────────────────────────────────
// Unittests — local one-shot HTTP server (same pattern as http.d)
// ────────────────────────────────────────────────────────────
version (unittest)
{
import std.socket;
import std.concurrency;
import std.string;
import std.format;
/// Spawn a one-shot TCP server that sends `response` to the
/// first connecting client then exits.
private static void oneShotResponder(shared TcpSocket listener,
string response) @trusted
{
try
{
auto sock = (cast() listener).accept();
// drain the request
ubyte[8192] buf = void;
sock.receive(buf[]);
sock.send(cast(immutable(ubyte)[]) response);
sock.close();
}
catch (Throwable) {}
}
/// Bind a listener on an ephemeral port, spawn a responder,
/// and return the URL.
private static auto bindAndSpawn(string response) @trusted
{
auto listener = new TcpSocket();
listener.bind(new InternetAddress("127.0.0.1",
InternetAddress.PORT_ANY));
listener.listen(1);
auto port = listener.localAddress().toPortString();
auto url = "http://127.0.0.1:" ~ port ~ "/";
spawn(&oneShotResponder, cast(shared) listener, response);
return url;
}
/// Build a minimal HTTP response string.
private static string httpResponse(int code, string reason, string body) @trusted
{
return format(
"HTTP/1.1 %d %s\r\nContent-Length: %d\r\n\r\n%s",
code, reason, body.length, body);
}
/// Create a Config that points zuurUrl at the given base URL.
private @safe Config testConfig(string baseUrl)
{
Config cfg;
cfg.zuurUrl = baseUrl;
cfg.cacheDir = "";
return cfg;
}
}
// ── Test (1): binary exists, version 2.1.0, constraint ge 2.0 → satisfies ──
@safe unittest
{
auto manifest = `return { name = "firefox", version = "2.1.0", summary = "Browser" }`;
auto baseUrl = bindAndSpawn(httpResponse(200, "OK", manifest));
auto cfg = testConfig(baseUrl);
auto constraint = DepConstraint.parse("firefox>=2.0");
auto result = checkBinaryVersion("firefox", constraint, cfg);
assert(result.exists, "binary should exist");
assert(result.ver == "2.1.0", "version should be 2.1.0, got: " ~ result.ver);
assert(result.satisfies, "2.1.0 should satisfy >=2.0");
}
// ── Test (2): binary exists, version 1.9, constraint ge 2.0 → !satisfies ──
@safe unittest
{
auto manifest = `return { name = "libfoo", version = "1.9", summary = "Library" }`;
auto baseUrl = bindAndSpawn(httpResponse(200, "OK", manifest));
auto cfg = testConfig(baseUrl);
auto constraint = DepConstraint.parse("libfoo>=2.0");
auto result = checkBinaryVersion("libfoo", constraint, cfg);
assert(result.exists, "binary should exist");
assert(result.ver == "1.9", "version should be 1.9, got: " ~ result.ver);
assert(!result.satisfies, "1.9 should NOT satisfy >=2.0");
}
// ── Test (3): 404 → exists:false, ver:"", satisfies:false ──
@safe unittest
{
auto baseUrl = bindAndSpawn(httpResponse(404, "Not Found", "gone"));
auto cfg = testConfig(baseUrl);
auto constraint = DepConstraint.parse("noexist>=1.0");
auto result = checkBinaryVersion("noexist", constraint, cfg);
assert(!result.exists, "binary should not exist for 404");
assert(result.ver == "", "version should be empty for 404");
assert(!result.satisfies, "non-existent binary should not satisfy");
}
// ── Test (4): 500 error → rethrows HttpException (not swallowed) ──
@safe unittest
{
auto baseUrl = bindAndSpawn(httpResponse(500, "Internal Server Error", "boom"));
auto cfg = testConfig(baseUrl);
bool caught = false;
try
{
auto constraint = DepConstraint.parse("brokenpkg>=1.0");
checkBinaryVersion("brokenpkg", constraint, cfg);
assert(false, "Expected HttpException for 500");
}
catch (HttpException e)
{
caught = true;
assert(e.msg.canFind("500"),
"Message should contain '500', got: " ~ e.msg);
}
assert(caught, "Should have thrown HttpException");
}
// ── Test (5): manifest without version field → BinaryException ──
@safe unittest
{
auto manifest = `return { name = "badpkg", summary = "no version field" }`;
auto baseUrl = bindAndSpawn(httpResponse(200, "OK", manifest));
auto cfg = testConfig(baseUrl);
bool caught = false;
try
{
auto constraint = DepConstraint.parse("badpkg>=1.0");
checkBinaryVersion("badpkg", constraint, cfg);
assert(false, "Expected BinaryException");
}
catch (BinaryException e)
{
caught = true;
assert(e.msg.canFind("cannot determine version"),
"Message should indicate missing version, got: " ~ e.msg);
}
assert(caught, "Should have thrown BinaryException");
}
// ── Test (6): version "2.1.0" with unconstrained dep (op none) → satisfies ──
@safe unittest
{
auto manifest = `return { name = "testpkg", version = "2.1.0", summary = "Test" }`;
auto baseUrl = bindAndSpawn(httpResponse(200, "OK", manifest));
auto cfg = testConfig(baseUrl);
auto constraint = DepConstraint.parse("testpkg");
assert(constraint.op == DepOp.none, "constraint should be unconstrained");
auto result = checkBinaryVersion("testpkg", constraint, cfg);
assert(result.exists, "binary should exist");
assert(result.ver == "2.1.0", "version should be 2.1.0, got: " ~ result.ver);
assert(result.satisfies, "unconstrained dep should always satisfy");
}