feat: observe upstream provider quota signals (#5211)
Codex and Claude already emit credential-level quota watermarks on ordinary
responses. CPA used to drop them. Keep the latest watermark in memory and
return it from the management auth-file API.
Hard rule: this is observation only. It must not change scheduling, cooldown
selection, or auth-file persistence.
Snapshot, not accumulation
- QuotaState now has ObservedAt and a bounded Signals map. MarkResult fills
them from the response headers already recorded on the request.
- Signals is the current response, not a union of earlier ones. Retry-After
and "limit reached" only appear on the response that produced them; merging
across responses would keep an expired value forever.
- A response with no quota header (transport failure, 5xx, unrelated endpoint)
leaves the previous snapshot in place.
- ObservedAt is the time of the current snapshot. It advances even when the
values did not change, so a consumer can tell a fresh reading from a stale
one.
- When two model states merge, keep the newer snapshot. Do not union keys
captured at different times.
What is observed, and what is not
- One predicate, ProviderSupportsQuotaObservation, decides the provider set.
- Keep Codex and Claude. Drop Kimi, xAI/Grok, Antigravity, and the Gemini
family (gemini/vertex/aistudio): their ordinary headers are not a reliable
credential-level remaining quota.
- Count-tokens reuses the credential but is not generation traffic.
ExecuteCount sets SkipQuotaObservation so those headers cannot replace the
last generation snapshot. Cooldown and success/failure accounting still run.
Cooldown must not overwrite the last snapshot
- Observation writes only ObservedAt and Signals.
- Cooldown writes only Exceeded, Reason, NextRecoverAt, and BackoffLevel,
through applyCooldownFields. Never assign a fresh QuotaState{...} over a
live value: that would zero the snapshot on 429, Cloudflare, credential-
scope sibling updates, and cooldown clears.
- If a credential-quota cooldown is still active, MarkResult still observes
an already-present model state. It does not create scheduler state just to
record a watermark.
- .cds files persist cooldownFieldsOf(Quota) only. Restore keeps the newer
ObservedAt, so reloading cooldown cannot clobber a newer in-memory snapshot.
- cooldownQuotaEqual still ignores observation fields, so a watermark change
cannot by itself persist cooldown or move the scheduler.
- The management payload omits every cooldown field, so it cannot be mistaken
for scheduler state or wired back into scheduling.
- Manual ResetQuota still clears the full QuotaState.
Codex websocket events
- Codex WS reports quota as codex.rate_limits frames, not HTTP headers.
ParseCodexQuotaEventHeaders turns one event into the same bounded header
shape, and MergeResponseHeaders folds it into the request-scoped holder.
additional_rate_limits is accepted as an object (websocket) or an array
(/wham/usage).
- Parse only through AppendCodexAPIWebsocketResponse. The shared
AppendAPIWebsocketResponse is also used by xAI, and xAI error frames really
do carry x-ratelimit-* headers. Parsing every frame as Codex quota would
forge Codex headers into another provider's request log.
- Also capture code_review_rate_limits.
- A malformed active-limit name drops only that one header, not the window
watermarks parsed from the same event.
- The type discriminator scans a bounded frame prefix, not every byte of
every frame.
- HTTP namespaces an extra limit by short name (x-codex-bengalfox-*); WS
namespaces it by limit name (GPT-5.3-Codex-Spark). The two paths cannot
emit the same header names. The X-Codex-Additional- prefix marks the WS
origin, and snapshot replacement keeps the two spellings from piling up.
Hardening
- Reject observed values with control characters. These strings reach the
plain-text request log, and Limit-Name is upstream-controlled, so CR/LF
could forge a header line.
- When the header cap is hit, keep plan/credits/primary ahead of
additional-limit namespaces, then sort names so truncation is deterministic.
- QuotaState.Clone deep-copies Signals and is used by Auth.Clone and
ModelState.Clone.
- Token stores still serialize credential metadata only, so observation adds
no auth-file writes. S
sususu98 committed
ca601db05d858472b25e7c56580555ab62b90e68
Parent: dc3c3b1
Committed by GitHub <noreply@github.com>
on 8/24/2026, 9:15:37 AM