bench: race busybox again, verify the wc oracle, average the speedup
Busybox wc is back in every suite, for shits and giggles. checkwc now walks PATH and identifies each wc by its --version answer (coreutils and fastwc respond; busybox names itself in the error it prints), so a wc symlinked to fastwc is detected and skipped instead of silently racing us against ourselves, and a missing coreutils is a clear configure error. Every run now ends with the average speedup of fastwc against coreutils and busybox, computed from the per-case ratios. The fail-fast verdict now requires a measurable (>0ms) reference time, so sub-millisecond cases stop flaking on startup noise, and the 100M line monster is pinned back to the coreutils oracle instead of whatever oracle was raced last.
This commit is contained in:
+16
-2
@@ -1,12 +1,22 @@
|
||||
#!/usr/bin/env bash
|
||||
# test-all.sh — compile the benchmark helper tools first, then run every
|
||||
# benchmark suite (words, lines incl. monsters, stdin). Exits non-zero if
|
||||
# any of them fails.
|
||||
# benchmark suite (words, lines incl. monsters, stdin) against every
|
||||
# oracle found (coreutils, plus busybox for shits and giggles). Exits
|
||||
# non-zero if any of them fails, and ends with the average speedup of
|
||||
# fastwc against each oracle.
|
||||
#
|
||||
# usage: ./test-all.sh
|
||||
set -u
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REPO_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
|
||||
# Share one ratios file across all suites so the final average covers
|
||||
# every case; each suite still races through its own process.
|
||||
RATIOS_FILE="$(mktemp /tmp/fastwc-ratios.XXXXXX)"
|
||||
export RATIOS_FILE
|
||||
|
||||
source "$SCRIPT_DIR/std.sh"
|
||||
|
||||
# 1. build the helper tools before running any benchmark so createtxt()
|
||||
# can use the fast C generator instead of the slow shell fallback
|
||||
@@ -29,6 +39,10 @@ rc_stdin=$?
|
||||
if [[ $rc_words -ne 0 || $rc_lines -ne 0 || $rc_stdin -ne 0 ]]; then
|
||||
printf '\ntest-all: FAILED (words=%s, lines=%s, stdin=%s)\n' \
|
||||
"$rc_words" "$rc_lines" "$rc_stdin" >&2
|
||||
print_averages
|
||||
rm -f "$RATIOS_FILE"
|
||||
exit 1
|
||||
fi
|
||||
printf '\ntest-all: all benchmarks passed\n'
|
||||
print_averages
|
||||
rm -f "$RATIOS_FILE"
|
||||
|
||||
Reference in New Issue
Block a user