perf: make gnu wc the slowest thing in the benchmark
SIMD kernels (AVX-512/AVX-2/SSE2, runtime-dispatched) counting a single pass over mmap'd files, split across cores past 8 MiB, with exact GNU oracle parity (NBSP included, glibc's decoder fixed, the whole -m path mirrored so counts agree at every boundary). Result: 1ms vs 2ms at 1M lines, 8-9ms vs 22-24ms at 10M. Forty years of dependencies, hand-tuned AVX-512 assembly, a translation team per language — and gnu wc still needs a buffer to copy into before it can count. We mapped the file and just counted. The benchmark suite no longer has a losing row; the shame report file is going to rust.
This commit is contained in:
@@ -11,7 +11,9 @@ up, it doesn't get faster — it gets *more dependencies*.
|
||||
fastwc is what `wc` looks like when nobody is paying you to maintain the
|
||||
museum. One file. One purpose. No translators. No gnulib. No AVX-512
|
||||
kernels hand-tuned by people whose entire job is compensating for the
|
||||
bloat around them. Just counting, correctly, at full speed.
|
||||
bloat around them — just our own: AVX-512, AVX-2, and SSE2 intrinsics
|
||||
with runtime dispatch, and a scalar SWAR fallback. Just counting,
|
||||
correctly, at full speed.
|
||||
|
||||
## The scoreboard
|
||||
|
||||
@@ -24,11 +26,14 @@ and exits non-zero. These are the facts:
|
||||
|-------|--------|
|
||||
| words (6 cases) | **6/6 wins.** Never slower, never wrong. |
|
||||
| lines (up to 100k lines) | **Wins.** GNU never sees us coming. |
|
||||
| lines (1M lines) | **GNU squeaks past by 1ms** — by shipping hand-tuned AVX-512 assembly written by a team of people who get paid for it. We call that cheating. Our SIMD pass is coming, and it will not be subtle. |
|
||||
| lines (10M lines) | Not yet run. The benchmark aborts at the first loss. Coward. |
|
||||
| lines (1M lines) | **Win: 1ms vs 2ms.** GNU's AVX-512 assist can't beat a mapped file. |
|
||||
| lines (10M lines) | **Win: 8-9ms vs 22-24ms (~2.5x).** GNU's lead never survives contact with the buffer. |
|
||||
| busybox lines (10M) | **Win: 8-9ms vs ~165ms (~18x).** If you must. |
|
||||
|
||||
The moment fastwc is slower than GNU `wc`, this project has failed and
|
||||
you should say so loudly in an issue. The benchmark is the contract.
|
||||
The how and why of the speed, with receipts, lives in
|
||||
[docs/PERFORMANCE.md](docs/PERFORMANCE.md).
|
||||
|
||||
## Why
|
||||
|
||||
@@ -37,8 +42,9 @@ you should say so loudly in an issue. The benchmark is the contract.
|
||||
- **GNU wc is slow where it should be fast.** Counting bytes is not
|
||||
supposed to be an architectural achievement.
|
||||
- **GNU wc counts like it's 1985** — because it is. We count like it's
|
||||
now: fixed-stride SWAR loops, lookup tables, zero function calls in the
|
||||
hot path.
|
||||
now: regular files are mapped and counted in parallel across cores,
|
||||
with SIMD kernels (AVX-512, AVX-2, SSE2) dispatched at runtime —
|
||||
zero function calls in the hot path.
|
||||
|
||||
## What it does
|
||||
|
||||
|
||||
Reference in New Issue
Block a user