fastwc/README.md

# fastwc

**The word counter GNU wishes it could be.**

GNU coreutils `wc` is the Apple of the FOSS world. Forty years of
accumulated bureaucracy wrapped in a binary. Translation teams. gettext.
`--help` output in fourteen languages. An autotools contraption the size
of a small city, all so you can count newlines. And when it can't keep
up, it doesn't get faster — it gets *more dependencies*.

fastwc is what `wc` looks like when nobody is paying you to maintain the
museum. One file. One purpose. No translators. No gnulib. No AVX-512
kernels hand-tuned by people whose entire job is compensating for the
bloat around them — just our own: AVX-512, AVX-2, and SSE2 intrinsics
with runtime dispatch, and a scalar SWAR fallback. Just counting,
correctly, at full speed.

## The scoreboard

The benchmark suite in `benchmarks/` races fastwc against GNU `wc` —
fail-fast. The moment we are slower, or disagree on a single count, it
writes a shame report and exits non-zero. These are the facts:

| Suite | Result |
|-------|--------|
| words (6 cases) | **6/6 wins.** Never slower, never wrong. |
| lines (up to 100k lines) | **Wins.** GNU never sees us coming. |
| lines (1M lines) | **Win: 1ms vs 2ms.** GNU's AVX-512 assist can't beat a mapped file. |
| lines (10M lines) | **Win: 8-9ms vs 22-24ms (~2.5x).** GNU's lead never survives contact with the buffer. |
| lines (100M lines) | **Win: ~70ms vs ~140ms.** The monster race. GNU gets lapped. |
| lines (1B lines) | **Solo, ~4-6s.** 11 GB in one pass; the only bottleneck left is the disk. |
| stdin words (1M lines) | **Win: 12x.** GNU still reads stdin like it's 1985. |
| stdin lines (10M lines) | **Win: ~2.5x.** We map stdin redirects; GNU maps nothing. |

The moment fastwc is slower than GNU `wc`, this project has failed and
you should say so loudly in an issue. The benchmark is the contract.
The how and why of the speed, with receipts, lives in
[docs/PERFORMANCE.md](docs/PERFORMANCE.md).

GNU wc's lone win — 1M lines, by a single millisecond, on hand-tuned
AVX-512 assembly — is a historical footnote now. We closed it, then the
10M case, then the 100M monster, and then we added a 1B-line race with
no opponent, because beating GNU wc at this point is just cruelty to
software. Busybox was removed from the suite for the same reason: it
stopped being a challenge and started being a participation trophy.

## Why

- **GNU wc is a dependency museum.** Its build needs gettext, gnulib,
  and a translator for every language on Earth. fastwc needs `cc`.
- **GNU wc is slow where it should be fast.** Counting bytes is not
  supposed to be an architectural achievement. Counting 11 GB of them
  in one pass, solo, while GNU's AVX-512 assembly still needs a buffer
  to copy into — that is.
- **GNU wc counts like it's 1985** — because it is. We count like it's
  now: regular files are mapped and counted in parallel across cores,
  with SIMD kernels (AVX-512, AVX-2, SSE2) dispatched at runtime —
  zero function calls in the hot path.

## What it does

```
fastwc [OPTION]... [FILE]...
fastwc [OPTION]... --files0-from=F
```

- `-l` lines, `-w` words, `-c` bytes, `-m` characters (multibyte),
  `-L` maximum line length (display width, tab stops every 8)
- `--files0-from=F` NUL-terminated file lists from F (or stdin when F is `-`)
- `--total=auto|always|only|never` control over the total row
- `--debug` report which line-counting acceleration is in use
- stdin, `-`, multiple files, `total` rows, GNU-compatible counts
- A CLI surface modeled on GNU coreutils `wc` 9.11: same options, same
  column alignment rule (counts are right-justified to the widest sum
  of regular file sizes), same multibyte word semantics, same error
  text. Swap it in and scripts keep working.
- POSIXLY_CORRECT is honored: the no-break space stops being a word
  separator, exactly like GNU wc.
- no `--help` in fourteen languages. One `--help`, in English, the
  language of people who ship software

## Build

Requires a C compiler and autotools. That's it. No gettext. No gnulib.
No translators.

```sh
./autogen.sh     # autoreconf -fi && ./configure
make
make release     # installs the release binary to bin/release/fastwc
```

## Benchmark

```sh
make bench                          # build release + run every suite
./benchmarks/test-all.sh            # all suites: words, lines, stdin
```

The suites live under `benchmarks/files/{lines,words}` (file input) and
`benchmarks/stdin/piping` (standard input), and fail fast the moment
fastwc loses a single case. The lines suite also carries the monsters:
100M lines raced against coreutils, and 1B lines timed solo — there is
no reference for that one; we are the reference now. The suites
interleave runs so both commands see identical cache warmth, keep the
minimum, and fail the moment fastwc loses a single case. GNU `wc` is
used as an oracle the same way you'd use a broken clock: occasionally
it's right, and it's the only one around.

## Development

The editor setup is one command:

```sh
make compile-commands   # compile_commands.json for clangd
```

clangd reads `.clangd`, `.clang-tidy`, and `.clang-format` — the style
guide, enforced by robots. We use `bear` when it's installed; the
fallback hand-rolls the single entry from the Makefile, because one
source file doesn't need a database.

- `make format` — make the code confess to the style guide
- `make format-check` — verify without touching
- `make lint` — clang-tidy, static analysis included

`.editorconfig` and `.gitattributes` keep every editor honest. Your
editor has opinions. So do we. Ours are in the repo.

## License

MIT. Do whatever you want. We're not GNU, we won't sue you — we'll just
be faster.

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) — the rules are the deal.
See [STYLEGUIDE.md](STYLEGUIDE.md) — the style is the law.