fix: thread-safety in count_sliced, ship measured wins
Review (5-lane) found one MAJOR: count_sliced ignored pthread_create's return value - on EAGAIN it joined an indeterminate pthread_t and summed an uninitialized slice, UB plus a silently wrong count. Threads are now initialized to 0, a failed create counts its slice inline, and nt is hard-capped at a named MAX_THREADS (the fixed jobs[8]/th[8] arrays smash the stack past 8 threads - reproduced by QA's thread sweep). Shipped from the measured optimization hunt: - -w-only mode skips the newline compare/popcount entirely (the (x-9)<5 range already covers '\n' in the whitespace mask) - a need_lines gate threads through every kernel, the scalar reference, the avx512 mirror, and the sliced workers. - Thread retune: 12 threads past 32 MiB, 16 past 256 MiB (was 8) - up to 18% on 110 MB, ~3% on 1.1 GB in QA's interleaved sweep. - Benchmark suites pin LC_ALL=C so GNU wc -w can't silently switch to multibyte decoding and inflate the win. - popcount16 is guarded to x86 builds (zero-warnings on other arches); checkwc's stray argument dropped. Verified: selftest (kernels + sliced, now with need_lines coverage), 120-trial fuzz vs GNU (file + stdin, both locales), full test-all suite three consecutive times, format-check and clang-tidy clean. The 1B solo monster now lands at 3.7-6s (up to 268 Mlines/s) depending on how warm the page cache is feeling.
This commit is contained in:
+8
-7
@@ -17,12 +17,13 @@ case, so every number below survived contact with the contract.
|
||||
| lines, 1M | 2-3ms | **1ms** |
|
||||
| lines, 10M | 21-24ms | **8-9ms** |
|
||||
| lines, 100M (monster) | ~140ms | **~70ms** |
|
||||
| lines, 1B (solo) | — | **~6s** |
|
||||
| lines, 1B (solo) | — | **~4-6s** |
|
||||
| bytes, 1GB sparse | reads all of it | `st_size`, no read |
|
||||
|
||||
That is a ~2.5x win over GNU on 10M lines, a 2x win on 1M lines, and
|
||||
a 2x win on the 100M monster. At 1B lines — 11 GB of data — the solo
|
||||
run lands around 6-8 seconds (125-170 Mlines/s), and the bottleneck is
|
||||
run lands around 4-6 seconds (200-270 Mlines/s, warm cache), and the
|
||||
bottleneck is
|
||||
honest to admit: an 11 GB file does not fit in the 15 GB of RAM this
|
||||
machine has, so the last monster is racing the disk. The 100M case,
|
||||
which fits, runs at ~17 GB/s, and that number is the counting.
|
||||
@@ -42,11 +43,11 @@ which fits, runs at ~17 GB/s, and that number is the counting.
|
||||
the data comes through stdin, but how we read it is our business.
|
||||
The stdin suite is why this shows up in the scoreboard too.
|
||||
3. **Parallel across cores.** Files over 8 MiB are split into 64-byte
|
||||
aligned slices counted by up to 8 threads. The kernels are pure, so
|
||||
the split needs no locks; word boundaries between slices are seeded
|
||||
from the byte before the slice, which makes the split exact. Below
|
||||
8 MiB the thread spawn would cost more than the counting, so we
|
||||
don't bother.
|
||||
aligned slices counted by up to 16 threads (12 past 32 MiB, 16 past
|
||||
256 MiB). The kernels are pure, so the split needs no locks; word
|
||||
boundaries between slices are seeded from the byte before the slice,
|
||||
which makes the split exact. Below 8 MiB the thread spawn would cost
|
||||
more than the counting, so we don't bother.
|
||||
4. **No work that isn't asked for.** `-c` on a regular file is
|
||||
`st_size` from `fstat` — GNU figured that one out too, so we copied
|
||||
the good idea. `-l` without `-w` skips the whitespace mask entirely.
|
||||
|
||||
Reference in New Issue
Block a user