perf: make gnu wc the slowest thing in the benchmark

SIMD kernels (AVX-512/AVX-2/SSE2, runtime-dispatched) counting a
single pass over mmap'd files, split across cores past 8 MiB, with
exact GNU oracle parity (NBSP included, glibc's decoder fixed, the
whole -m path mirrored so counts agree at every boundary).

Result: 1ms vs 2ms at 1M lines, 8-9ms vs 22-24ms at 10M. Forty years
of dependencies, hand-tuned AVX-512 assembly, a translation team per
language — and gnu wc still needs a buffer to copy into before it
can count. We mapped the file and just counted. The benchmark suite
no longer has a losing row; the shame report file is going to rust.
This commit is contained in:
2026-08-29 15:32:08 -04:00
parent 5fc2f3e668
commit ff465e981b
6 changed files with 859 additions and 64 deletions
+11 -5
View File
@@ -11,7 +11,9 @@ up, it doesn't get faster — it gets *more dependencies*.
fastwc is what `wc` looks like when nobody is paying you to maintain the
museum. One file. One purpose. No translators. No gnulib. No AVX-512
kernels hand-tuned by people whose entire job is compensating for the
bloat around them. Just counting, correctly, at full speed.
bloat around them — just our own: AVX-512, AVX-2, and SSE2 intrinsics
with runtime dispatch, and a scalar SWAR fallback. Just counting,
correctly, at full speed.
## The scoreboard
@@ -24,11 +26,14 @@ and exits non-zero. These are the facts:
|-------|--------|
| words (6 cases) | **6/6 wins.** Never slower, never wrong. |
| lines (up to 100k lines) | **Wins.** GNU never sees us coming. |
| lines (1M lines) | **GNU squeaks past by 1ms** — by shipping hand-tuned AVX-512 assembly written by a team of people who get paid for it. We call that cheating. Our SIMD pass is coming, and it will not be subtle. |
| lines (10M lines) | Not yet run. The benchmark aborts at the first loss. Coward. |
| lines (1M lines) | **Win: 1ms vs 2ms.** GNU's AVX-512 assist can't beat a mapped file. |
| lines (10M lines) | **Win: 8-9ms vs 22-24ms (~2.5x).** GNU's lead never survives contact with the buffer. |
| busybox lines (10M) | **Win: 8-9ms vs ~165ms (~18x).** If you must. |
The moment fastwc is slower than GNU `wc`, this project has failed and
you should say so loudly in an issue. The benchmark is the contract.
The how and why of the speed, with receipts, lives in
[docs/PERFORMANCE.md](docs/PERFORMANCE.md).
## Why
@@ -37,8 +42,9 @@ you should say so loudly in an issue. The benchmark is the contract.
- **GNU wc is slow where it should be fast.** Counting bytes is not
supposed to be an architectural achievement.
- **GNU wc counts like it's 1985** — because it is. We count like it's
now: fixed-stride SWAR loops, lookup tables, zero function calls in the
hot path.
now: regular files are mapped and counted in parallel across cores,
with SIMD kernels (AVX-512, AVX-2, SSE2) dispatched at runtime —
zero function calls in the hot path.
## What it does