test: record core topology and support CPU pinning in the benchmark (#291)
Closes #290.
## Why
`scripts/bench-local-interval.sh` reports CPU as percent of one core, and the whole point of its design (fixed 200x50 terminal, environment block, CPU-time delta instead of `ps -o %cpu`) is that results from different machines are comparable. On a heterogeneous CPU that does not hold, because "one core" is not one quantity. Identical work costs a different amount of CPU time on a performance core than on an efficiency core, and nothing in the output recorded which one ran.
Measured on an NVIDIA GB10 (Cortex-X925 at 3.9GHz plus Cortex-A725 at 2.8GHz), pinning the same run to one cluster or the other moves the result by about 1.5x. That is larger than the interval effect the script exists to measure, which was +19.4% on the same host. Intel P/E hybrids and Apple Silicon have the same property, so the existing M5 Max reference numbers carry it too.
## Changes
**`topology` line in the environment block.** Groups logical CPUs by maximum frequency, which separates ARM big.LITTLE clusters and Intel P/E cores alike, falling back to the device-tree `cpu_capacity` on ARM systems without cpufreq, and to `hw.nperflevels` on macOS. When none is readable it reports `unknown` rather than assuming uniform, since assuming uniform is the specific error this is meant to prevent. Uniform hosts print just count and speed; heterogeneous hosts additionally print each cluster's CPU list, range-collapsed so it can be pasted straight into the new flag.
**`-c CPUS` pins the run** via `taskset`, and the choice appears on a new `affinity` line. An unpinned run on a heterogeneous host now says so explicitly instead of quietly reporting a placement-dependent number. Rejected on macOS with an explanation rather than silently ignored, because the kernel exposes no userspace CPU affinity control and a run genuinely cannot be pinned to P or E cores there. Invalid lists are validated by handing them to `taskset` itself rather than reimplementing range syntax.
**`-r COUNT` repeats each configuration** and reports mean and standard deviation, because a single window on a low-cost host puts the effect close to the run-to-run spread. Default stays 1, so existing invocations keep both their runtime and their exact output format; the mean-and-deviation format appears only when repeats are requested. `measure` was split into `measure_once` plus an aggregating wrapper, with failures carried through exit codes so a partial run reports how many windows succeeded rather than silently averaging fewer.
## Output
```
cpu Cortex-X925 + Cortex-A725 (20 cores)
topology heterogeneous: 10x 3.90GHz (cpus 5-9,15-19), 10x 2.81GHz (cpus 0-4,10-14)
affinity cpus 0-4,10-14 (taskset)
window 20s measured after 8s warmup, 2 repeats
=== results (percent of one core) ===
default cpu= 1.58% +/- 0.04 (n=2) cpu_time=0.32s / 20s rss=31MB
-i 2s cpu= 1.58% +/- 0.04 (n=2) cpu_time=0.32s / 20s rss=31MB
-i 3s cpu= 1.40% +/- 0.07 (n=2) cpu_time=0.28s / 20s rss=30MB
```
## Review findings, all fixed in later commits
Implementation and security review each found real defects, every one reproduced on hardware before being fixed.
**The affinity line could lie.** `taskset` does not validate a CPU list the way the first commit's comment claimed: it fails only when *no* CPU in the list exists, so it rejects `99-200` but silently accepts `0,99` and narrows it to CPU 0. The line then advertised `cpus 0,99` for a run pinned to one core, which is worse than printing nothing, since this line exists precisely so the context travels with the number. The effective mask is now read back after launch and reported, with the requested list shown alongside when they differ.
**Exact-key grouping over-split real hardware.** This machine's `cpu_capacity` reads 718, 731, 997, 1017, and 1024 across a two-cluster part, so the fallback reported five tiers, one of them a single CPU. Intel parts with per-core turbo binning have the same shape on the cpufreq path. Keys are now bucketed within 5% of the group's fastest member, compared against that fixed representative so small steps cannot drift a bucket across a real boundary, and the range collapser sorts numerically since a bucket can merge keys whose CPU runs interleave.
**The measured PID was a race.** It came from diffing `pgrep -x all-smi` snapshots and taking the first new PID. `-c` made that likely rather than theoretical, since the natural way to use it is one pinned run per cluster: running both at once made them report byte-identical numbers, cpu_time included, while each `affinity` line attested to a different placement. The PID now comes from `tmux list-panes -F '#{pane_pid}'`, the process tmux actually started. Two concurrent pinned runs now report distinct numbers in the expected direction.
**Nothing cleaned up on interrupt**, so Ctrl-C left an orphan rendering a 200x50 TUI forever, competing for CPU with the next run and, under `-c`, for exactly the pinned cluster. A trap now kills this invocation's sessions. The INT and TERM handlers exit rather than only cleaning up, because a bash trap returns to the point of interruption, so cleanup alone killed the window in flight and then opened the next one.
**Unvalidated input reached the shell command string.** `-i '$(touch /tmp/x)1'` created the file and a bare `-i '*'` glob-expanded so filenames became intervals; `-d 0.5` was accepted silently while being shorter than the clock resolution, understating CPU roughly twofold. Both are whole seconds by definition and are now checked, with globbing off for the interval walk.
Also: tmux stderr is surfaced on launch failure instead of a bare "could not start", and the window is timed with bash 5's `EPOCHREALTIME` where available, since whole-second truncation at both ends puts up to 1.7% of error on a 60s window, landing directly on the spread `-r` exists to expose.
## Tests
Validated on the GB10 host across unpinned, performance-pinned, and efficiency-pinned runs at 8s, 10s, 20s, and 30s windows with 1 and 2 repeats.
Pinning demonstrably does what it is for. Pinned to the efficiency cluster, `default` and `-i 2s` agreed **exactly** at 1.58%, where they should be identical because local mode resolves to 2s on Linux, and the deviation tightened to about 0.04 from the 0.08 to 0.13 measured unpinned.
Topology detection was exercised against the shipped awk for uniform x86, Intel P/E (8P+16E), Intel favoured-core binning, the real ARM capacity values, single core, non-contiguous clusters, three genuine tiers, and CPU indices past 9. The 5% bucket merges binned siblings without merging real tiers.
Interrupt handling was tested through a real terminal Ctrl-C rather than a background kill: bash sets SIGINT to ignored for asynchronous jobs and a signal ignored at entry cannot be trapped, which made the first test runs look like a script bug when the harness was at fault. Under a genuine Ctrl-C the session and child both go away.
Flag validation is covered: `-r 0`, `-r abc`, `-d abc`, `-d 0`, `-i '$(touch ...)'`, `-i '1;id'`, `-i '*'`, and an out-of-range `-c 99-200` are each rejected with a specific message.
`shellcheck` 0.11.0 is clean at default severity. The `-r 1` success-line format string is byte-identical to the pre-PR version, so existing reports stay comparable.
Shell only, no Rust touched, so the existing suite is unaffected and nothing about what `all-smi` collects or reports changes, satisfying #290's fourth acceptance criterion.
Also documents the comparability limit, all flags, and the container caveat for `-c` under Testing in `DEVELOPERS.md`, along with the `libdrm-dev` build prerequisite on Linux. J
Jeongkyu Shin committed
02c5e5b258a37998969e25ad39a72b39d3289ef0
Parent: cfd9289
Committed by GitHub <noreply@github.com>
on 7/27/2026, 8:42:46 AM