readme, roadmap: say what is true now

The status block no longer says scaffold, "The loop" no longer says none of it
runs yet, and the table of what each verb actually runs names format-patch
rather than a bare git diff.

Configuration is real: the file and its keys, the two environment variables,
init, and a doctor transcript showing what got resolved and from where. What is
left is Later and Far, plus three gaps named rather than papered over: real
mail, a full kernel build, and the fix round trip against a real kernel patch.

The tests section describes the fixture harness and the three tests that want
the host's own tools.

ROADMAP's Now and Next record what landed and what each item does now. The
series paragraph explains why the sidecar records a whole range rather than a
base: a re-roll that regenerated base..HEAD would pick up every commit that
landed on the branch since create and mail it with the set.

README.md

@@ -2,11 +2,13 @@
 
 A monolithic kernel work wrapper, to make it easy.
 
-> **Status: scaffold.** The command surface parses, every verb is wired
+> **Status: the loop closes.** Every verb below runs. The chain from a random
-> through `main` to its module, and the paths resolve. Every body is a
+> open bug to a patch that `git send-email` accepts has been walked end to end,
-> `todo!()` stub. Nothing builds a kernel, boots qemu, scrapes bugzilla, or
+> including the v2 re-roll threaded off the original. `kernel test` builds a
-> sends mail yet. What exists is the shape of the tool and the seams to fill
+> tree and boots it under qemu, `doctor` says what is missing before you start,
-> in, and *Where to pitch in* below is the list.
+> and `init` writes the config file. `Next` in `ROADMAP.md` has landed; what is
+> left is `Later` and `Far`, plus the three gaps named under *What is not
+> proven* below.
 
 ## Why
 
@@ -32,10 +34,12 @@ What each verb actually ends up running:
 |---|---|
 | `patch check` / `patch format` | `scripts/checkpatch.pl` **from your tree** |
 | `patch commit` | `git commit` |
-| `patch create` | `git diff <base>` |
+| `patch create` | `git format-patch` |
+| `patch update` | a rename, or `git format-patch --reroll-count` for a series |
 | `patch submit` | `scripts/get_maintainer.pl` **from your tree**, then `git send-email` |
 | `kernel test` | `make`, then `qemu-system-x86_64` |
-| `kernel quest` | HTTP against `bugzilla.kernel.org` |
+| `kernel quest` | the bugzilla.kernel.org REST API |
+| `doctor` | git, the tree's own scripts, qemu, and the paths |
 
 checkpatch.pl and get_maintainer.pl are worth calling out. They come from
 the tree you are working in, so spectral cannot drift out of date with the
@@ -49,17 +53,19 @@ the command yourself.
 
 ## The loop
 
-What the finished tool should feel like. None of this runs yet:
+What the finished tool feels like. Every line below has been run:
 
 ```console
 $ spectral kernel quest
 12345 usb: xhci: device does not enumerate after resume
+component: Drivers/USB
 https://bugzilla.kernel.org/show_bug.cgi?id=12345
 
 # ... go and fix it, in the tree ...
 
 $ spectral patch check
 $ spectral patch format          # checkpatch --fix, then check again
+$ git add -A                     # `patch commit` commits what you staged
 $ spectral patch commit "usb: xhci: re-arm the port after resume" --signoff
 $ spectral kernel test           # build, then boot it under qemu
 $ spectral patch create 000-xhci-port-rearm
@@ -75,11 +81,35 @@ $ spectral patch update 000-xhci-port-rearm.patch -v 2
 $ spectral patch submit v2-000-xhci-port-rearm.patch --in-reply-to '<msg-id-of-v1>'
 ```
 
+Staging is yours on purpose. `patch commit` runs `git commit`, so what goes in
+is what you put there; spectral will not sweep a dirty tree into your patch,
+because a stray file in a kernel patch is a review comment you did not want.
+
+For a set rather than a single patch:
+
+```console
+$ spectral patch create xhci-port-rearm --range master..HEAD \
+      --cover-letter "Re-arm the port after resume"
+~/.spectral/patches/xhci-port-rearm
+$ spectral patch submit xhci-port-rearm --dry-run
+$ spectral patch update xhci-port-rearm     # -v is inferred; every file moves
+```
+
+The series directory holds the patches plus a `series.toml` recording the range
+it was generated for (both ends as hashes), the revision, the file list and the
+cover letter, which is what lets a re-roll happen without retyping the range.
+Re-rolling regenerates exactly that range: a commit that landed on the branch
+since `create` does not join the set, because a re-roll should not mail commits
+nobody asked to send. The blurb's first line becomes the cover letter's subject.
+Leave `--cover-letter` off and the set is written with git's template subject in
+it, which `git send-email` refuses: an unsendable series is better than a cover
+letter nobody wrote.
+
 One deviation from the obvious design: `patch update` is a single subcommand
 taking `-v N` rather than a generated `update-v2` / `update-v3`. clap
 subcommands are static, and a flag beats a subcommand you cannot tab-complete.
 Leave `-v` off and it infers the next revision from the file's current `vN-`
-prefix.
+prefix, or from the sidecar for a series.
 
 ## Install
 
@@ -108,26 +138,58 @@ You also need:
 
 ## Configuration
 
-There is no config file yet. Two paths matter, and both come from the
+Two paths and a couple of settings, resolved in this order: the environment,
-environment with `$HOME`-relative defaults:
+then `~/.config/spectral/config.toml`, then the built-in default.
 
-| what | resolution |
+| what | environment | file key | default |
-|---|---|
+|---|---|---|---|
-| kernel tree | `$SPECTRAL_KERNEL`, else `~/.spectral/linux` |
+| kernel tree | `$SPECTRAL_KERNEL` | `kernel_tree` | `~/.spectral/linux` |
-| patch dir | `~/.spectral/patches` |
+| patch dir | `$SPECTRAL_PATCH_DIR` | `patch_dir` | `~/.spectral/patches` |
+| always CC | | `always_cc` | none |
+| send-email identity | | `send_email.from` | git's own `user.email` |
 
 ```console
 $ export SPECTRAL_KERNEL=$HOME/src/linux
+$ spectral init --tree $HOME/src/linux      # or --clone <url> [--depth N]
+wrote /home/you/.config/spectral/config.toml
+tree: /home/you/src/linux
 ```
 
+`init` verifies the tree before it writes anything, and refuses to overwrite a
+config that points somewhere else unless you pass `--force`. Run it twice with
+the same tree and the second run says so and changes nothing.
+
 A tree is accepted only if it has `scripts/checkpatch.pl`. Otherwise you get
 `not a kernel source tree` rather than a confusing failure three steps later.
 A missing tree reports the path and the variable that would have set it. The
 tree is not needed for `kernel quest`, which only wants the network.
 
-`~/.config/spectral/config.toml` and a `spectral init` to clone the tree are
+A path in the config file has to be absolute, and an unknown key is an error
-planned, not built. `src/config.rs` is the only file that has to change to
+rather than a shrug: a mistyped setting that silently does nothing is worse
-add either.
+than one that stops the run and says which line is wrong.
+
+Run `spectral doctor` to see what got resolved and from where:
+
+```console
+$ spectral doctor
+ok   kernel tree             /home/you/src/linux (from the config file)
+ok   kernel tree shape       scripts/checkpatch.pl is there
+ok   checkpatch.pl           scripts/checkpatch.pl (the only thing that decides kernel style)
+ok   get_maintainer.pl       scripts/get_maintainer.pl (the only thing that knows who to mail)
+ok   MAINTAINERS             the maintainer map the script reads
+ok   git                     git version 2.55.0
+ok   git send-email          runs
+ok   sender identity         [email protected]
+warn mail route              no sendemail.* config: --dry-run works, a real send will not
+                             -> configure `git send-email` before dropping --dry-run
+ok   qemu                    QEMU emulator version 11.1.1
+ok   patch directory         /home/you/.spectral/patches (exists, the built-in default)
+ok   config file             /home/you/.config/spectral/config.toml
+doctor: nothing failed, 1 warned
+```
+
+A warning keeps the exit code at 0; a failed check exits 1 and the summary names
+the first one, because that is the one to fix first.
 
 ## Commands
 
@@ -140,75 +202,81 @@ Usage: spectral <COMMAND>
 Commands:
   kernel  Find work, build it, boot it
   patch   Carry a change from working tree to mailing list
+  doctor  Check the tree, the scripts, the mail setup and the paths
+  init    Point the config at a kernel tree, verifying or cloning one
   help    Print this message or the help of the given subcommand(s)
 ```
 
-`spectral kernel quest` · `kernel test`
+`spectral kernel quest`, `kernel test`
 
-`spectral patch check` · `format` · `commit` · `create` · `submit` · `update`
+`spectral patch check`, `format`, `commit`, `create`, `submit`, `update`
 
-Every one of them has `--help` that says more than this README does, and for
+`spectral doctor`, `init`
-now a body that panics with a description of what it is supposed to do.
+
+Every one of them has `--help` that says more than this README does.
+
+## What is not proven
+
+Named rather than papered over, because a thin wrapper is only worth trusting
+where it says where it stops:
+
+- **Real mail is unverified.** No patch from this project has been sent to a
+  mailing list by spectral. The acceptance evidence is `git send-email
+  --dry-run` reaching `Dry-OK` with the recipients spectral chose.
+- **A full kernel build is not part of the test suite.** `kernel test`'s build
+  step is exercised against a Makefile that writes the image; the boot is
+  exercised against a real qemu with a real `bzImage`, but nothing here builds
+  a kernel from source, because that is hours of work that proves nothing about
+  this crate.
+- **`patch format`'s fix round trip is proven against a made-up patch**, not
+  against a real kernel posting whose damage checkpatch can repair.
+
+## Tests
+
+```console
+$ cargo test --locked
+```
+
+No test needs a kernel tree or a network, and none needs `git send-email`
+configured. Each one builds a throwaway kernel-shaped git repository in a temp
+directory, points `HOME`, `SPECTRAL_KERNEL` and git's global config at it, and
+drives the built binary through `CARGO_BIN_EXE_spectral`. The fixture's `PATH`
+and git config are the test's, not the machine's, which is how a missing qemu, a
+missing sender identity or a missing `git send-email` is produced deliberately
+instead of inherited.
+
+One host tool is assumed: git with its `send-email` support. The `doctor` tests
+wrap it so a machine without that support still passes, and the send-email flow
+tests use the real one, because that is the tool under test.
+
+Three tests do want the host's own tools, and they are ignored unless asked for:
+
+```console
+$ SPECTRAL_REAL_CHECKPATCH=1 SPECTRAL_REAL_TREE=/usr/src/linux \
+      cargo test --locked -- --ignored real_checkpatch
+$ SPECTRAL_REAL_GETMAINTAINER=1 SPECTRAL_REAL_TREE=/usr/src/linux \
+      cargo test --locked -- --ignored real_get_maintainer
+$ SPECTRAL_LIVE=1 cargo test --locked -- --ignored live_bugzilla
+```
+
+They are where a kernel-version change to checkpatch's summary line, or to
+get_maintainer's role words, or to bugzilla's REST answer, shows up as a test
+failure rather than inside someone's patch workflow.
 
 ## Where to pitch in
 
-Every stub is a small function that already has its signature, its doc
+The verbs are written and the loop closes. What is left is in `ROADMAP.md`:
-comment, and its caller in `main`. Pick one and you only need to read the
+`Later` holds more quest sources, more checks, review tracking and packaging;
-file it lives in.
+`Far` holds the TUI and `spectral auto`.
 
-Roughly in order of how much they unblock.
+Two of those are more useful than they look. `kernel/quest.rs` is behind the
+`QuestSource` trait, so a syzbot source is one impl and no change to the
+command. `patch/series.rs` holds the sidecar a series carries, and review
+tracking is the feature that will want to add to it.
 
-### patch submit
+The three gaps at the end of *What is not proven* are honest small tasks: a
-
+real send, a real kernel build under `kernel test`, and the fix round trip
-The most self-contained win, and the only one that needs no network. It does
+against a real kernel patch.
-need a kernel tree and a working `git send-email`.
-
-- `patch/maintainers.rs` · `lookup` runs `get_maintainer.pl --git` over the
-  patch, splits the output into `To:` and `Cc:`, and collects the files the
-  patch touches.
-- `patch/maintainers.rs` · `add_cc` folds `--cc` flags in without duplicates.
-- `patch/mod.rs` · `submit` ties those two together with `git send-email`,
-  with `--dry-run` stopping one step short.
-
-### patch check, format, commit, create
-
-The everyday verbs. `patch/checkpatch.rs` · `run` already has the `Target`
-enum (working tree, a revision, or a patch file) and `Report` with
-`is_clean`. What is missing is the process call and pulling the error and
-warning counts out of checkpatch's output.
-
-### patch update
-
-Small, and worth writing a test for. `patch/mod.rs` · `reroll_path` holds the
-naming rule in one pure function: strip one leading `vN-`, then prepend
-`v<revision>-`. Running it twice at the same revision should do nothing,
-which is easier to assert than to explain.
-
-### kernel quest
-
-The fun one if you like HTML. `kernel/quest.rs` · `Bugzilla::fetch_open` is
-the only place `reqwest` and `scraper` earn their place in `Cargo.toml`, and
-`run` then applies `--filter` and picks one issue. The `QuestSource` trait is
-where you would add syzbot or a lore.kernel.org thread later.
-
-### kernel test
-
-Needs a machine you are willing to boot kernels on. `kernel/qemu.rs` ·
-`build` then `boot`. Streaming serial output as it arrives beats buffering it
-until qemu exits, and it is the difference between a boot hang you can
-diagnose and one you cannot.
-
-### git plumbing
-
-`git.rs` · `current_branch`, `diff_against`, `commit`, `rev_parse`. Each is a
-few lines over the `run` that is already written, which makes these the
-easiest place to start.
-
-There are no tests yet. The naming rule in `reroll_path` and the recipient
-split in `lookup` are the two that most want one.
-
-Longer-term work, and the things spectral deliberately will not do, are in
-`ROADMAP.md`.
 
 ## Development
 
@@ -216,9 +284,10 @@ Longer-term work, and the things spectral deliberately will not do, are in
 $ cargo fmt --all
 $ cargo clippy --all-targets -- -D warnings
 $ cargo test
+$ cargo doc --no-deps
 ```
 
-All three are clean on `main` and are the bar for a change.
+All four are clean on `main` and are the bar for a change.
 
 A few conventions:
 
@@ -228,12 +297,16 @@ A few conventions:
 - Command modules return what they produced and `main` prints it, so output
   formatting lives in one place.
 - Errors go through the one `Error` enum and `?`. Nothing in the command
-  paths calls `unwrap`.
+  paths calls `unwrap`. An external command that fails reports what it said,
-- A new stub gets a `todo!()` naming the command it will end up running. A
+  from stdout as well as stderr, because git puts some refusals on stdout.
-  module that is unreachable until its caller exists carries a single
+- Every module that shells out has a comment saying why that tool and not
-  `#[allow(dead_code)]` with the reason attached. There are no crate-wide
+  another API: `checkpatch.pl` and `get_maintainer.pl` come from the tree, and
-  allows, so the warnings come back as the stubs get filled in.
+  `git format-patch` does the series naming because that is already its job.
-- No `unsafe` and no FFI.
+- No `unsafe` and no FFI. `std::env::set_var` is `unsafe` under edition 2024,
+  which is why environment-dependent behaviour is tested through the binary
+  rather than in a unit test.
+- No `todo!()` is left in the command path. If you add one, say in the test
+  suite what will prove it gone.
 
 ## License
 

ROADMAP.md

@@ -6,61 +6,73 @@ ranges from one afternoon to a design conversation first.
 
 ## Now
 
-Implement the stubs. `README.md` lists them grouped by how much they
+Done. Every verb runs, and the loop closes end to end: a commit went from a
-unblock, and `patch submit` is where most people should start, since it is
+working tree to a `git send-email` plan, then round again as v2 threaded off
-the only verb that needs no network.
+the original. `README.md` has the walkthrough and the acceptance evidence.
 
-The milestone worth having is the loop closing:
+The milestone was this:
 
 ```text
 quest -> edit -> check -> format -> commit -> create -> submit -> v2
 ```
 
 One patch through one review cycle without dropping to a shell to run git or
-checkpatch by hand. Until that works, nothing else on this page is worth
+checkpatch by hand. The one step that stays yours on purpose is `git add`:
-building.
+`patch commit` commits what you staged, because sweeping a dirty tree into a
+kernel patch is a review comment nobody asked for.
 
-Two tests belong here too, both named in the README: the `vN-` naming rule in
+The two tests this section named are in place: the `vN-` naming rule in
-`reroll_path`, and the `To:`/`Cc:` split in `lookup`.
+`reroll_path`, and the `To:`/`Cc:` split in `lookup`, the second of which is
+also asserted against the host's real `get_maintainer.pl` in an opt-in test.
+
+`rg -n 'todo!' src/` finds nothing. What is left overall is the three gaps in
+the README's *What is not proven*, and `Later` and `Far` below.
 
 ## Next
 
+All three landed, in this order. The sections below say what each one does now.
+
 ### A series instead of a single patch
 
-Everything today produces one file. Real submissions are often a patch set
+Done. `patch create xhci-port-rearm --range master..HEAD --cover-letter
-with a cover letter, which means:
+"<blurb>"` writes a numbered set through `git format-patch`, cover letter
+included; `patch submit` takes a directory or several files and sends them in
+one `git send-email` invocation; `patch update <dir>` re-rolls the whole set
+through `--reroll-count`, so every file moves to vN together, and the
+superseded files go only once the new revision is complete. The blurb's first
+line becomes the cover letter's subject, and leaving `--cover-letter` off
+leaves git's template in place, which `git send-email` refuses.
 
-- `patch create` writing a numbered set from a commit range, and a
+A `series.toml` in the directory records the range it was generated for: both
-  `0000-cover-letter.patch` alongside it
+ends as hashes, the revision, the files, and the blurb. The range is what makes
-- `patch submit` taking several files at once, which `git send-email`
+a re-roll safe, since regenerating `base..HEAD` instead would quietly pick up
-  already supports
+every commit that landed on the branch since `create` and mail it with the set.
-- `patch update` re-rolling the whole set, so every file in it moves to v2
-  together, the way `git format-patch --reroll-count` does it
-- per-patch version numbers, for the same reason
-
-This is the widest gap between what spectral does and how kernel patches are
-actually sent, so it comes first once the stubs are done.
 
 ### spectral doctor
 
-The README asks for three things before spectral is useful: a kernel tree,
+Done. It checks, in the order a first run hits them: the tree path and where it
-`git send-email` that works, and `qemu-system-x86_64` on `$PATH`. Today you
+came from, that the path is a kernel tree, `scripts/checkpatch.pl`,
-find out that `send-email` was never configured at the moment you try to
+`scripts/get_maintainer.pl`, `MAINTAINERS`, git, `git send-email`, a sender
-submit, which is the worst time to learn it.
+identity, whether any mail route is configured, qemu on `$PATH`, that the patch
+directory is writable, and whether a config file exists.
 
-A `doctor` subcommand would check the tree, the two scripts inside it, the
+One line per check with the fix under it, and the summary names the first hard
-`send-email` config, qemu, and the resolved paths, then name whichever one is
+failure. Exit 1 when something hard failed; a missing qemu or a missing mail
-missing. The `Error` variants for the tree cases already exist.
+route warns and keeps exit 0, because `--dry-run` is where the loop actually
+stops on this machine.
 
 ### Config file and init
 
-`src/config.rs` carries the TODO. Two additions:
+Done. `~/.config/spectral/config.toml` (or `$XDG_CONFIG_HOME`), holding the
+tree path, the patch directory, always-CC addresses and a send-email identity.
+Precedence is the environment over the file over the built-in default, and
+`$SPECTRAL_PATCH_DIR` joined `$SPECTRAL_KERNEL` for symmetry.
 
-- `~/.config/spectral/config.toml` holding the tree path, the patch
+`spectral init --tree <path>` verifies a tree, or `--clone <url> [--depth N]`
-  directory, your send-email identity, and any addresses you always want
+fetches one, then writes the config. A second run with the same tree says so
-  CC'd. The environment variable keeps winning over the file.
+and writes nothing; a run that would change an existing answer is refused until
-- `spectral init`, to clone a tree or point at one you already have and
+`--force`. Malformed TOML, an unknown key and a relative path are named errors
-  verify it, so nobody has to read the Configuration section twice.
+rather than surprises three commands later.
 
 ## Later
 
@@ -93,7 +105,8 @@ re-roll.
 
 That needs spectral to record what it sent and where, which nothing does
 today. Where that state lives is the design question to settle before the
-feature.
+feature. A series already carries `series.toml`, which is the smallest version
+of that record: it is what a re-roll reads.
 
 ### Packaging and polish
 
@@ -164,7 +177,8 @@ should not produce a v3. `reroll_path` is already specified as a no-op at the
 same revision, and `auto` inherits that requirement.
 
 It depends on the verbs, on `doctor` to fail fast instead of halfway, and on
-the config and state files the sections above describe.
+the config and state files the sections above describe. `doctor` and the config
+file are here now; the state file is what review tracking still has to decide.
 
 ## Not planned