Busybox wc is back in every suite, for shits and giggles. checkwc now
walks PATH and identifies each wc by its --version answer (coreutils and
fastwc respond; busybox names itself in the error it prints), so a wc
symlinked to fastwc is detected and skipped instead of silently racing
us against ourselves, and a missing coreutils is a clear configure
error. Every run now ends with the average speedup of fastwc against
coreutils and busybox, computed from the per-case ratios.
The fail-fast verdict now requires a measurable (>0ms) reference time,
so sub-millisecond cases stop flaking on startup noise, and the 100M
line monster is pinned back to the coreutils oracle instead of whatever
oracle was raced last.
Sub-millisecond runs showed as a flat 0ms, hiding real differences. The
suite now times fastwc with a time_us helper, keeps the minimum reading
in µs, derives ms from it, and prints both (fastwc: 0ms (767µs)). GNU wc
stays at ms precision; solo case shows µs too.
Review (5-lane) found one MAJOR: count_sliced ignored pthread_create's
return value - on EAGAIN it joined an indeterminate pthread_t and summed
an uninitialized slice, UB plus a silently wrong count. Threads are now
initialized to 0, a failed create counts its slice inline, and nt is
hard-capped at a named MAX_THREADS (the fixed jobs[8]/th[8] arrays
smash the stack past 8 threads - reproduced by QA's thread sweep).
Shipped from the measured optimization hunt:
- -w-only mode skips the newline compare/popcount entirely (the
(x-9)<5 range already covers '\n' in the whitespace mask) - a
need_lines gate threads through every kernel, the scalar reference,
the avx512 mirror, and the sliced workers.
- Thread retune: 12 threads past 32 MiB, 16 past 256 MiB (was 8) -
up to 18% on 110 MB, ~3% on 1.1 GB in QA's interleaved sweep.
- Benchmark suites pin LC_ALL=C so GNU wc -w can't silently switch to
multibyte decoding and inflate the win.
- popcount16 is guarded to x86 builds (zero-warnings on other arches);
checkwc's stray argument dropped.
Verified: selftest (kernels + sliced, now with need_lines coverage),
120-trial fuzz vs GNU (file + stdin, both locales), full test-all suite
three consecutive times, format-check and clang-tidy clean. The 1B
solo monster now lands at 3.7-6s (up to 268 Mlines/s) depending on how
warm the page cache is feeling.
Suites split into benchmarks/files/{lines,words} and
benchmarks/stdin/piping, with the monsters bolted onto the lines suite:
100M lines raced against coreutils (~2x win), 1B lines solo (~6-8s,
11 GB in one pass). Stdin redirects from regular files are now mmap'd
in count_stream, so the stdin suite wins too — up to 12.00x on words.
The ratio column now reports how many times faster fastwc is, not how
much of GNU's time it used. busybox was removed from the suite: it
stopped being a challenge and started being a participation trophy.
GNU wc's lone win — 1M lines by one millisecond on hand-tuned AVX-512
assembly — is now a historical footnote, and the README says so.
huntedbytheirs