feat(http): add synchronous HTTP client with std.net.curl
This commit is contained in:
@@ -0,0 +1,35 @@
|
||||
Warning
|
||||
Warning ## Warning for package tofu ##
|
||||
Warning
|
||||
Warning The following compiler flags have been specified in the package description
|
||||
Warning file. They are handled by DUB and direct use in packages is discouraged.
|
||||
Warning Alternatively, you can set the DFLAGS environment variable to pass custom flags
|
||||
Warning to the compiler, or use one of the suggestions below:
|
||||
Warning
|
||||
Warning warningsAsErrors: Use "buildRequirements" to control the warning level
|
||||
Warning
|
||||
Generating test runner configuration 'tofu-test-application' for 'application' (executable).
|
||||
Warning Excluding main source file src/main.d from test.
|
||||
Starting Performing "unittest" build using /usr/bin/dmd for x86_64.
|
||||
Up-to-date toml 1.0.0: target for configuration [library] is up to date.
|
||||
Up-to-date tofu ~main: target for configuration [tofu-test-application] is up to date.
|
||||
Finished To force a rebuild of up-to-date targets, run again with --force
|
||||
Running tofu-test-application
|
||||
Warning: malformed TOML config at /tmp/tofu-test-config-bad-183825.toml: Invalid table key declaration (2:0)
|
||||
Warning: invalid TOFU_DEFAULT_JOBS 'not-a-number', using default 1
|
||||
5 modules passed unittests
|
||||
Warning
|
||||
Warning ## Warning for package tofu ##
|
||||
Warning
|
||||
Warning The following compiler flags have been specified in the package description
|
||||
Warning file. They are handled by DUB and direct use in packages is discouraged.
|
||||
Warning Alternatively, you can set the DFLAGS environment variable to pass custom flags
|
||||
Warning to the compiler, or use one of the suggestions below:
|
||||
Warning
|
||||
Warning warningsAsErrors: Use "buildRequirements" to control the warning level
|
||||
Warning
|
||||
Starting Performing "debug" build using /usr/bin/dmd for x86_64.
|
||||
Up-to-date toml 1.0.0: target for configuration [library] is up to date.
|
||||
Building tofu ~main: building configuration [application]
|
||||
Linking tofu
|
||||
Finished To force a rebuild of up-to-date targets, run again with --force
|
||||
@@ -108,3 +108,75 @@ _Auto-scaffolded by /start-work. Append new entries below - never overwrite._
|
||||
- **`BuildResult.failed` as `BuildFailure[]`**: the plan describes it as `struct {string name; string reason}[]` — in D this is a named struct `BuildFailure` with array field `BuildFailure[] failed`. The `failedNames()` helper extracts just the names for caller convenience.
|
||||
- **`DepConstraint.parse` is a static factory method**: returns a new `DepConstraint` by value, never allocates on the heap. `@safe` throughout.
|
||||
- **All structs are `@safe`**: no methods do I/O, no `@system` calls. The module stands alone — no imports from other tofu modules.
|
||||
|
||||
---
|
||||
|
||||
## Task 6 — `tofu.http` (sync HTTP client via std.net.curl)
|
||||
|
||||
### std.net.curl API (key members for HTTP struct)
|
||||
- **Construction**: `HTTP(url)` via `opCall`, or `HTTP()` + `.url = url` setter.
|
||||
The `.url` setter auto-prepends `http://` if no scheme is present.
|
||||
- **Timeouts** (all are `@property` setters taking `core.time.Duration`):
|
||||
- `connectTimeout` — connect phase only (`CurlOption.connecttimeout_ms`).
|
||||
- `operationTimeout` — DNS + connect + transfer total (`CurlOption.timeout_ms`).
|
||||
- `dataTimeout` — low-speed activity timeout (NOT total read timeout;
|
||||
sets `low_speed_limit` + `low_speed_time`).
|
||||
- For a "read 120s" requirement use `operationTimeout = 120.seconds` —
|
||||
it covers the entire operation including data transfer.
|
||||
- **Redirects**: `maxRedirects` (property setter, takes `uint`).
|
||||
Defaults to 10. Set to 0 to disable, `uint.max` for infinite.
|
||||
Internally sets `CurlOption.followlocation` + `CurlOption.maxredirs`.
|
||||
- **User-Agent**: `setUserAgent(string)` — instance method (not property).
|
||||
- **onReceive**: `void delegate(ubyte[])` returning `size_t`. Must accept
|
||||
all bytes or the request aborts. Use `return data.length;`.
|
||||
- **onReceiveStatusLine**: Not a direct property on HTTP struct — must
|
||||
be set via the `onReceiveHeader` property setter which wraps the
|
||||
delegate and internally fires `onReceiveStatusLine` when it detects
|
||||
an HTTP status line. After `perform()` the final status is available
|
||||
via `http.statusLine`.
|
||||
- **statusLine**: `HTTP.StatusLine` struct with `.code` (`ushort`),
|
||||
`.majorVersion`, `.minorVersion`, `.reason`. Reset to zero before
|
||||
each `perform()`.
|
||||
- **perform()**: `CurlCode perform(ThrowOnError = Yes.throwOnError)`.
|
||||
Default throws on curl errors, but the thrown exception classes are:
|
||||
- `CurlTimeoutException : CurlException` — operation timed out (code 28).
|
||||
- `CurlException` — all other curl errors.
|
||||
- `HTTPStatusException : CurlException` — only thrown by high-level
|
||||
wrappers (_basicHTTP); low-level `perform()` does NOT throw on
|
||||
HTTP status codes — you must check `statusLine.code` yourself.
|
||||
- **Exception isolation pattern**: catch `CurlTimeoutException` and
|
||||
`CurlException` in `@trusted` helpers, re-throw as `HttpException`
|
||||
(which extends `Exception`, not `CurlException`). Callers never see
|
||||
curl types.
|
||||
- **verifyPeer / verifyHost**: On by default in HTTP struct (no change needed).
|
||||
|
||||
### @safe / @trusted architecture
|
||||
- All public functions (`get`, `downloadFile`) are `@safe`.
|
||||
- The `configure(HTTP)`, `getImpl`, and `downloadFileImpl` helpers are
|
||||
`@trusted` — they are the ONLY places `std.net.curl` (which is
|
||||
`@system`-heavy) is used.
|
||||
|
||||
### Download pattern (.part temp file + rename)
|
||||
- Ported from ZETA `lib/fetch.lua`: write to `dest.part`, rename on
|
||||
success, remove partial on error. `scope(failure)` at function
|
||||
level handles all error paths automatically.
|
||||
- `File(tmpPath, "wb")` — binary write mode. File destructor auto-closes;
|
||||
explicit `f.close()` needed before rename.
|
||||
|
||||
### Unittest local server
|
||||
- Used `std.concurrency.spawn` + `std.socket.TcpSocket` for a
|
||||
self-contained one-shot HTTP server (no external dependency on Python3).
|
||||
- Pattern: bind ephemeral port (`InternetAddress.PORT_ANY`), spawn thread
|
||||
that `accept()`s one connection, sends canned HTTP response, exits.
|
||||
`cast(shared)` the listener to pass to `spawn`. Mark test helpers
|
||||
`@trusted`.
|
||||
- HTTP response must include `Content-Length` header or libcurl hangs.
|
||||
|
||||
### Connection refused test
|
||||
- Bind socket, record port, close it, then `get()` to that port.
|
||||
curl returns error 7 (`CURLE_COULDNT_CONNECT`) → `CurlException`
|
||||
→ caught and re-thrown as `HttpException`.
|
||||
|
||||
### Build verified
|
||||
- `dub build` passes with warnings-as-errors.
|
||||
- `dub test` passes — all 5 modules (config, log, types, vercmp, http).
|
||||
|
||||
Reference in New Issue
Block a user