SIGN IN SIGN UP

test: add a portable local-mode collection cost benchmark (#289)

Adds the benchmark tooling issue #288 asks for. Refs #288, and deliberately does not close it: #288 is satisfied by *results* from Linux and Windows hardware, which this PR cannot produce. It lands the instrument so those results are collectable and comparable.

## Why

PR #286 did two things, only one of which is cross-platform. It moved local mode's default collection interval from 3s to 1s on Apple Silicon and to 2s everywhere else, and it replaced the IOReport sampling strategy so a collection went from ~495ms to ~24ms. The sampling half is gated to macOS (`macos_native` is `#[cfg(target_os = "macos")]`), so Linux, Windows, and Intel Mac took the 1.5x polling increase with no offsetting reduction. Every number behind that decision came from one Apple Silicon machine.

## What the script does

`scripts/bench-local-interval.sh` measures `all-smi local` CPU across the default configuration plus a configurable set of explicit intervals.

```
$ scripts/bench-local-interval.sh -h        # usage
$ cargo build --release --bin all-smi
$ scripts/bench-local-interval.sh           # 60s window, intervals 1/2/3
$ scripts/bench-local-interval.sh -d 120 -i "2 5"
```

Two design points are what make results from different reporters worth comparing:

**Fixed terminal size.** The real TUI runs detached in tmux at 200x50. Render cost scales with terminal size, so an unfixed size would make two machines' numbers incomparable for reasons that have nothing to do with the collection path.

**CPU-time delta, not `ps -o %cpu`.** That column is a decaying recent average on macOS and a lifetime-since-start average on Linux. Reading it would produce two different quantities on the two platforms the script targets. The script reads the process CPU-time delta over the window instead: `/proc/PID/stat` (utime + stime over `CLK_TCK`) on Linux, `ps -o cputime=` on macOS, where BSD `ps` carries centisecond resolution that Linux's truncates away. Results are percent of one core.

An environment block (OS, kernel, CPU, core count, GPU, process count, all-smi version) prints above the numbers, because collection cost depends on which device readers are active and results without that context cannot be interpreted.

## Reference run

Apple M5 Max, 18 cores, 1035 processes, release build, 60s window after 8s warmup:

```
  default        cpu=  5.28%   cpu_time=3.17s / 60s   rss=20MB
  -i 1s          cpu=  5.44%   cpu_time=3.32s / 61s   rss=21MB
  -i 2s          cpu=  3.28%   cpu_time=1.97s / 60s   rss=20MB
  -i 3s          cpu=  2.40%   cpu_time=1.44s / 60s   rss=19MB
```

`default` matching `-i 1s` within noise independently confirms #286's interval fix is in effect on Apple Silicon. The 3s to 2s step, which is what non-Apple-Silicon platforms absorbed, costs about +37% on this cost profile. Whether it costs the same where the reader mix is NVML plus AMD plus Intel rather than IOReport is exactly the open question.

## Note on earlier numbers

Two corrections to what was reported on #286 and #287, both already applied to those threads:

- The benchmarking hardware was an Apple M5 Max, not an M1 Ultra. The M1 Ultra is the machine in the screenshots that started the investigation; I conflated it with the machine the benchmarks ran on.
- Those numbers used `ps -o %cpu`. On the same machine the CPU-time-delta method in this script reports a somewhat higher absolute figure for the same configuration, so the absolute values in those threads are indicative rather than exact. The before/after comparisons used one consistent method throughout and are unaffected.

## Linux run, and a fix it turned up

The Linux path has since been exercised on an NVIDIA GB10 (DGX Spark), Ubuntu 24.04.4, aarch64, 20 cores. Full numbers are on #288; the summary is that the `/proc/PID/stat` path is correct (validated against a busy loop pinned at one full core, where it reported 100.01%), `default` matches `-i 2s` to within 3.2% so #286's interval fix is confirmed live on Linux, and the 3s to 2s step costs +19.4% there against +37% on the M5 Max, which is 0.14s of CPU per minute or 0.0117% of that machine.

That run exposed a defect, fixed in the second commit. `detect_cpu` read `model name` from `/proc/cpuinfo`, an x86-only field, so on aarch64 the awk matched nothing, printed an empty string, and still exited 0, which meant the trailing `|| echo unknown` never fired and the environment block reported a blank CPU. Since GB10, GH200, and Grace are all aarch64, the platform this benchmark most needs results from was the one silently dropping its CPU identity. It now falls back to `lscpu`, joining the distinct `Model name` values so a heterogeneous part reads `Cortex-X925 + Cortex-A725`.

One limit of the metric is worth stating rather than leaving implicit, and is tracked separately as #290. On a heterogeneous CPU, "percent of one core" is not one quantity: pinning the same run to the GB10's X925 cluster versus its A725 cluster moves the result by about 1.5x, which is larger than the interval effect this script exists to measure. Ratios within a single host stay robust (+17.9% pinned to performance cores, +21.4% to efficiency cores, +19.4% unpinned), so conclusions drawn from them hold, but absolute percentages are comparable across machines only when core placement is stated. Apple Silicon P/E and Intel hybrid parts have the same property, so the M5 Max reference numbers above carry it too.

## Tests

Shell only, no Rust touched, so the existing suite is unaffected. `shellcheck` clean as of the first commit; it is not installed on the GB10 host, so the second commit's change was not re-linted there, and it declares `local name` before assigning so it does not introduce SC2155. Exercised on macOS at 15s, 20s, and 60s windows, and on Linux at 10s and 60s windows across four configurations and three repeats.

Building on Linux requires `libdrm-dev`: without it the link fails on `-ldrm` and `-ldrm_amdgpu` even on a host with no AMD GPU, because `libamdgpu_top` is a hard dependency of the glibc Linux target.

Also documents usage under Testing in `DEVELOPERS.md`.
J
Jeongkyu Shin committed
cfd9289b573b6431c673c15c3759167c2ec2ea6c
Parent: 18fb595
Committed by GitHub <noreply@github.com> on 7/27/2026, 7:19:31 AM