ci: run the nightly benchlab experiment
Three jobs: plan resolves the experiment once, measure fans out over a matrix of package shards, publish merges and commits. plan resolving the commits centrally is what makes bench-nightly's shard-agreement check meaningful -- shards that disagree about head, baseline or prev are not commensurable, and the merged record would be a fiction. Sharding is per package. Each shard measures all three arms for its own benchmarks, so on-machine comparability is preserved while wall-clock is divided; benchmarks are charted independently, so cross-package comparability was never needed. Only publish joins the `benchmarks` concurrency group, at job level rather than workflow level. Holding the group for the hours the measurement takes would block the branch's other writers for no reason. Shards upload artifacts rather than pushing, so N shards cannot race each other on the benchmarks branch. A failed shard costs its own benchmarks rather than the whole night, and the merged record lists the shards it actually contains. Raw benchmark output is archived too: it is the canonical Go format, and it is what makes a night re-analysable at a different alpha without re-measuring. Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
S
Stephan Renatus committed
90c8a7bd6bc9f05aac3bf7466d18c60e9af953b9
Parent: 2d6a096
Committed by Stephan Renatus <s_renatus@apple.com>
on 9/1/2026, 7:04:21 AM