SIGN IN SIGN UP

ci: run the nightly benchlab experiment

Three jobs: plan resolves the experiment once, measure fans out over a matrix of
package shards, publish merges and commits.

plan resolving the commits centrally is what makes bench-nightly's
shard-agreement check meaningful -- shards that disagree about head, baseline or
prev are not commensurable, and the merged record would be a fiction.

Sharding is per package. Each shard measures all three arms for its own
benchmarks, so on-machine comparability is preserved while wall-clock is
divided; benchmarks are charted independently, so cross-package comparability
was never needed.

Only publish joins the `benchmarks` concurrency group, at job level rather than
workflow level. Holding the group for the hours the measurement takes would
block the branch's other writers for no reason.

Shards upload artifacts rather than pushing, so N shards cannot race each other
on the benchmarks branch. A failed shard costs its own benchmarks rather than
the whole night, and the merged record lists the shards it actually contains.
Raw benchmark output is archived too: it is the canonical Go format, and it is
what makes a night re-analysable at a different alpha without re-measuring.


Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
S
Stephan Renatus committed
90c8a7bd6bc9f05aac3bf7466d18c60e9af953b9
Parent: 2d6a096
Committed by Stephan Renatus <s_renatus@apple.com> on 9/1/2026, 7:04:21 AM