feat(core,webapp,run-engine): stamp a shard key onto run, batch and waitpoint ids (#4788)
## Summary Adds the id-minting half of sharding run data across several databases. Every entity that co-locates with a run now carries the run's shard key inside its own id, so its row is routable on its own instead of needing a directory table or a scatter across shards. Nothing changes for users yet. With no shard descriptors configured, every mint path produces exactly the ids it produces today, and the trigger path issues no extra query. ## Design A run's mint target travels as a single object carrying the kind and, when sharded, the shard character. The shard and the caller's region both occupy index 24 of a run-ops id, so passing them together makes it impossible for a caller to set two competing sources for one slot. A child run, a batch and a batch item read the shard from their parent's id rather than resolving a fresh one, so a run tree never splits across databases. Three services carried that branch separately, and one had already drifted, so it now lives in one function. Waitpoints mint through one shared pure function used by both the webapp and the run engine. They have to agree byte for byte, because the routing store refuses a waitpoint whose id is not stamped for the shard it is being written to: ```ts mintWaitpointIdForShard(key) // standalone token: the environment's shard mintWaitpointIdFor(anchorId) // co-located: the anchor's shard, or a cuid ``` The core is always freshly minted rather than derived from the anchor, since a derived body would be byte-identical to the run's own id. One latent bug fixed on the way: the failed-run path duplicated the mint branch inline and had drifted, so a child of a sharded parent would have been written to a different database from its parent. ## Guarding the create sites The expensive failure here is a waitpoint minted without its anchor's shard: one of the five create sites writes through a path that has no stamp check, so a miss there strands a blocked run with nothing logged. An enumerated census plus a source scan fails when a new create site appears, when an existing one stops passing its anchor, or when a site is added to a file the scan does not yet cover. The census was written before any site was converted, so it went red on the first commit and green as the last site landed. Both holes an earlier draft had, a file-granular count and a scan that missed the directory these mints used to live in, were confirmed closed by reintroducing them and watching the guard fail. ## Before enabling a shard Merging this is inert: with the mint list empty the resolver returns before it reads anything, and ids are identical to a measured `main` baseline. Verified against a live shard locally, including that the resolver issues no query across thirty triggers with no shard configured. Enabling is gated on two other pull requests, both open, both by the same author, each of which owns the file involved: - **#4781** adds the gen-2 shard arm to read-through. Without it a gen-2 run cannot wait on a token at all: the wait route resolves the waitpoint through read-through, which is shard-blind, so the wait fails. Do not set the mint list before it merges. - **#4780** generalises the distinct-database sentinel. Without it a shard pointed at the same physical database as the gen-1 store boots without complaint, which voids the disjointness the fan-out sums rely on. Testing also turned up a silent read-path gap that neither pull request covers: the paths that hydrate runs from ClickHouse through a fixed pair of Postgres clients drop gen-2 rows on the floor, so the runs list would show fewer rows than its own count with nothing logged. That needs its own change before a shard carries real traffic, and it is filed as such. ## Notes for reviewers Four commits in the middle of the stack do not typecheck in isolation: a signature change and its call-site repairs are separate commits, so bisecting inside the stack needs care. Commit `845ab06` also understates itself, since it rewrites the primary trigger path's mint alongside the failed-run path it names. No changeset and no server-changes entry: every path is inert while the feature is off, so there is nothing to tell users yet. --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
D
Daniel Sutton committed
15dd973f92d588152cb671325dccf269021b9f92
Parent: 4e00651
Committed by GitHub <noreply@github.com>
on 8/27/2026, 2:57:32 PM