huntedbytheirs/fastwc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: honest scoreboard — startup note, us methodology, current hardware

Browse Source
This commit is contained in:
huntedbytheirs
2026-09-09 05:23:54 -04:00
parent 3cde946a68
commit 0bb53e9c15
3 changed files with 174 additions and 77 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+73 -33
View File
@@ -17,34 +17,65 @@ correctly, at full speed.
## The scoreboard
The benchmark suite in `benchmarks/` races fastwc against GNU `wc` —
fail-fast. The moment we are slower, or disagree on a single count, it
writes a shame report and exits non-zero. These are the facts:
The benchmark suite in `benchmarks/` races fastwc against GNU `wc`,
busybox `wc`, and toybox `wc` — fail-fast. The moment we are slower, or
disagree on a single count, it writes a shame report and exits
non-zero. These are the facts. Earlier numbers were measured on a
workstation with more cores than taste; these are from an ordinary
laptop APU, which is where the rest of you live:
| Suite | Result |
|-------|--------|
| words (6 cases) | **6/6 wins.** Never slower, never wrong. |
| lines (up to 100k lines) | **Wins.** GNU never sees us coming. |
| lines (1M lines) | **Win: 1ms vs 2ms.** GNU's AVX-512 assist can't beat a mapped file. |
| lines (10M lines) | **Win: 8-9ms vs 22-24ms (~2.5x).** GNU's lead never survives contact with the buffer. |
| lines (100M lines) | **Win: ~70ms vs ~140ms.** The monster race. GNU gets lapped. |
| lines (1B lines) | **Solo, ~4-6s.** 11 GB in one pass; the only bottleneck left is the disk. |
| stdin words (1M lines) | **Win: 12x.** GNU still reads stdin like it's 1985. |
| stdin lines (10M lines) | **Win: ~2.5x.** We map stdin redirects; GNU maps nothing. |
| words, tiny (1-10k lines) | **6/6 vs coreutils, busybox, and toybox.** Correct on every one — but these races finish in the low hundreds of microseconds and prove *right*, not *fast*. See the note on startup below. |
| words, 1M lines (11 MB) | **1.6 ms vs coreutils 7.9 ms (4.9x).** Busybox 10.0x (16.2 ms), toybox 9.9x (15.9 ms). |
| words, 10M lines (110 MB) | **5.1 ms vs coreutils 66 ms (13.1x).** Busybox 27.0x (160 ms), toybox 15.9x (157 ms). |
| lines, 1M (11 MB) | **1.6 ms vs busybox 16.9 ms (10.4x)**, toybox 13.3x (17.6 ms). vs coreutils this is startup, not throughput — see below. |
| lines, 10M (110 MB) | **4.6 ms vs coreutils 5.8 ms (1.27x).** GNU throws every thread at a mapped file and the scoreboard doesn't move. Busybox 20.1x (160 ms), toybox 31.3x (154 ms). |
| lines, 100M monster | **29.2 ms vs coreutils 47.5 ms (1.6x).** The bigger the file, the wider the gap. |
| lines, 1B solo | **266 ms.** 11 GB in one pass at 41.4 GB/s — 3.76 billion lines per second. We are the reference now. |
| stdin lines, 10M | **4.6 ms vs coreutils 5.9 ms (1.28x).** Busybox 16.6x (161 ms), toybox 16.8x (154 ms). |
| stdin words, 1M | **1.5 ms vs coreutils 7.1 ms (4.7x).** GNU reads stdin like it's 1985. Busybox 12.7x, toybox 10.4x. |
The moment fastwc is slower than GNU `wc`, this project has failed and
you should say so loudly in an issue. The benchmark is the contract.
The how and why of the speed, with receipts, lives in
Measured on an AMD Ryzen AI 7 PRO 350 — 8 cores / 16 threads, boost up
to 5.09 GHz, 384 KiB L1d / 256 KiB L1i / 8 MiB L2 / 16 MiB L3 — with
64 GiB of DDR5-4800 dual-channel (2x 32 GiB SODIMM). Opponents: GNU
coreutils 9.11, busybox 1.36.1, toybox 0.8.13. Page-cache-warm files,
three interleaved runs per case, minimum kept, both sides timed at
microsecond resolution by a C timer (`benchmarks/tools/timeit`) — no
`date`-fork rounding. Cases where the reference itself finishes in
under 5 ms are startup, not throughput: fastwc must still match the
count, and the case is reported, but it is excluded from the averages.
Raced cases allow a 2% dead-heat margin so a genuine tie can't flake
on scheduler jitter. All of it, exactly as `benchmarks/` prescribes.
The moment fastwc is slower than any of them, this project has failed
and you should say so loudly in an issue. The benchmark is the
contract. The how and why of the speed, with receipts, lives in
[docs/PERFORMANCE.md](docs/PERFORMANCE.md).
GNU wc's lone win — 1M lines, by a single millisecond, on hand-tuned
AVX-512 assembly — is a historical footnote now. We closed it, then the
10M case, then the 100M monster, and then we added a 1B-line race with
no opponent, because beating GNU wc at this point is just cruelty to
software. Busybox is back in the suite too — for shits and giggles. It
stopped being a challenge years ago, but watching it count is the
closest thing this project has to a hobby. Every run ends with the
average speedup against each oracle, so the cruelty is quantified.
Toybox joined the suite for the same reason busybox is still there:
shits and giggles. It stopped giggling around the first average —
~15x — and it has never won a single case; counting 10M lines takes
it 16 times as long as us, and busybox 27. GNU coreutils is the only
opponent that keeps score, and even its threaded counter only manages
a dead heat on mid-size files — never a win, and the moment the file
stops fitting in a polite buffer, the dead heat stops being polite.
A note on startup, in the interest of honesty: on a one-line file the
whole race happens in the low hundreds of microseconds — fastwc
~0.25 ms, GNU ~0.33 ms, toybox ~0.37 ms, busybox ~0.14 ms. Microseconds
either way. Nobody will ever notice a difference that small, and it
does not matter in the bigger picture: the tiny cases in the table are
here to prove fastwc is never *wrong*, not to brag about a head start
that evaporates the moment the page cache warms up. That is why the
suite now times both sides at microsecond resolution and files anything
the reference finishes in under 5 ms under "startup-bound": correct,
reported, and excluded from the averages — because nobody should be
racing startup, least of all a word counter. The cases that matter are
the ones where counting takes longer than starting — and those are the
ones in this table. Every run ends with the average speedup against
each oracle — coreutils ~4.5x, busybox ~13-15x, toybox ~15-18x — so
the cruelty is quantified.
## Why
@@ -100,17 +131,26 @@ make bench # build release + run every suite
```
The suites live under `benchmarks/files/{lines,words}` (file input) and
`benchmarks/stdin/piping` (standard input), and fail fast the moment
fastwc loses a single case. Every suite races against GNU coreutils wc
and, when it is installed, busybox wc. The benchmark locates the real
oracles by their `--version` answer — a `wc` that answers as fastwc is a
symlink somebody made, and is skipped rather than raced against itself.
The lines suite also carries the monsters: 100M lines raced against
coreutils, and 1B lines timed solo — there is no reference for that
one; we are the reference now. The suites interleave runs so both
commands see identical cache warmth, keep the minimum, and fail the
moment fastwc loses a single case. When the run ends, the average
speedup of fastwc against each oracle is printed.
`benchmarks/stdin/piping` (standard input). Every suite races fastwc
against GNU coreutils wc and, when it is installed, busybox wc and
toybox wc. The benchmark locates the real oracles by their `--version`
answer — a `wc` that answers as fastwc is a symlink somebody made, and
is skipped rather than raced against itself.
Both sides are timed at microsecond resolution by `tools/timeit`, a C
timer built on the spot (`test-all.sh` compiles it next to `genfile`)
that forks the command, discards its output, and reads the clock after
it exits — no `date`-fork rounding. The suites interleave runs so both
commands see identical cache warmth and keep the minimum of three.
A case where the reference itself finishes in under 5 ms is startup,
not throughput: fastwc must still match the count, but the case is
reported as `startup-bound` and excluded from the averages. Raced
cases allow a 2% dead-heat margin so a genuine tie can't flake — but a
real loss writes a shame report, fails the suite, and exits non-zero.
The words suite grows to 1M and 10M lines; the lines suite carries the
monsters: 100M lines raced against coreutils, and 1B lines timed solo
— there is no reference for that one; we are the reference now. When
the run ends, the average speedup of fastwc against each oracle is
printed.
## Development