docs: honest scoreboard — startup note, us methodology, current hardware

This commit is contained in:
2026-09-09 05:23:54 -04:00
parent 3cde946a68
commit 0bb53e9c15
3 changed files with 174 additions and 77 deletions
+22 -15
View File
@@ -10,18 +10,25 @@ These aren't guidelines. They're the deal.
### 1. The benchmark is the contract
`./benchmarks/test-all.sh` must pass. Not "mostly pass." Not "pass on
your machine." Pass. Every suite — words, lines (monsters included),
and stdin — races fastwc against GNU wc.
your machine." Pass. Every suite — words (growing to 10M lines), lines
(monsters included), and stdin — races fastwc against GNU coreutils
wc, busybox wc, and toybox wc at microsecond resolution.
The moment fastwc is slower than GNU wc — or disagrees with it on a
single count — your change does not ship. We did not spend this much
effort being faster than a forty-year-old dependency museum just so you
could add a `strlen()` in the hot loop.
The moment fastwc is slower than any of them on a real race — or
disagrees with any of them on a single count — your change does not
ship. We did not spend this much effort being faster than a
forty-year-old dependency museum just so you could add a `strlen()` in
the hot loop.
The old standing exception — the 1M-line case, where GNU's hand-tuned
AVX-512 assembly squeaked out a one-millisecond win — is closed. We
beat them there too now (see docs/PERFORMANCE.md). There are no
exceptions left, and there is no excuse to be slower anywhere else.
The benchmark is honest about what it measures. Both sides are timed
by a C timer, and a case the reference finishes in under 5 ms is
startup, not throughput: fastwc must still count it right, but the
case is reported as startup-bound and excluded from the averages.
Raced cases allow a 2% dead-heat margin so a genuine tie can't flake
on scheduler jitter. That margin exists because GNU's threaded counter
occasionally dead-heats with us on mid-size files — it has never won a
raced case, and no exception is carved out for it
(see docs/PERFORMANCE.md).
### 2. C99, or don't bother
@@ -68,11 +75,11 @@ feature needs a new file, your feature is over-engineered.
### 7. Tests are the benchmarks
There is no separate test suite, because the benchmark *is* the test
suite — it checks correctness against GNU wc (the only oracle around,
and it's wrong often enough to keep us humble) and speed in the same
breath. Add a flag? It gets benchmarked. Change the counting? It gets
benchmarked. The fail-fast scripts in `benchmarks/` are your tests, and
all of them must pass before you open a PR.
suite — it checks correctness against GNU coreutils wc, busybox wc,
and toybox wc (all three are wrong often enough to keep us humble) and
speed in the same breath. Add a flag? It gets benchmarked. Change the
counting? It gets benchmarked. The fail-fast scripts in `benchmarks/`
are your tests, and all of them must pass before you open a PR.
### 8. No AI slop