perf: make gnu wc the slowest thing in the benchmark

SIMD kernels (AVX-512/AVX-2/SSE2, runtime-dispatched) counting a
single pass over mmap'd files, split across cores past 8 MiB, with
exact GNU oracle parity (NBSP included, glibc's decoder fixed, the
whole -m path mirrored so counts agree at every boundary).

Result: 1ms vs 2ms at 1M lines, 8-9ms vs 22-24ms at 10M. Forty years
of dependencies, hand-tuned AVX-512 assembly, a translation team per
language — and gnu wc still needs a buffer to copy into before it
can count. We mapped the file and just counted. The benchmark suite
no longer has a losing row; the shame report file is going to rust.
This commit is contained in:
2026-08-29 15:32:08 -04:00
parent 5fc2f3e668
commit ff465e981b
6 changed files with 859 additions and 64 deletions
+9 -5
View File
@@ -41,17 +41,21 @@ designed it — it was. We're one file, one purpose, one opinion.
## The hot path
`count_stream()`, `count_newlines()`, `count_words()` are the product.
The rest of the file is just the packaging.
`count_stream()`, `count_newlines()`, `count_words()`, and the SIMD
kernels (`count_lw_avx512`, `count_lw_avx2`, `count_lw_sse2`) are the
product. The rest of the file is just the packaging.
- **Fixed stride.** No per-token function calls. No `isspace()` in a
loop — that's what the lookup table is for.
- **No allocation, no locks, no syscalls in the counting loop.** The
`fread` is the only syscall, and it's not yours to add to.
kernels are pure; parallel slices need no locks. The `fread` (or the
single `mmap`) is the only syscall, and it's not yours to add to.
- **Branchless where it costs nothing.** A predictable branch is fine;
a mispredicted one is a lie you told the CPU.
- **The buffer is `static`, 128 KiB, and never grows.** GNU's wc reads
in chunks too — ours just doesn't make a ceremony of it.
- **Regular files are mapped, not streamed.** The kernel hands us the
pages; we count them, split across cores for anything past 8 MiB.
Stdin and odd files fall back to the `static`, 128 KiB buffer that
never grows.
## Counts