AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.
chore(backend): seed block preflight estimates from 30d prod usage (#13074)
## Why
PR #13031 landed the pre-flight estimate machinery for dynamic-cost
blocks (`SECOND` / `ITEMS` / `COST_USD`) but ships an empty
`block_preflight_estimates.json` — every dynamic-cost block still
pre-charges 0. This PR populates the JSON so the platform pre-charges a
historical mean at block start, and post-flight reconciliation
(`charge_reconciled_usage`) only has to settle a small delta instead of
the full cost.
Shrinks the `billing_leak` surface: today, if a user's wallet empties
between block start and reconciliation, the entire dynamic cost is
uncollectable. With this seed in place, the bulk is collected up front,
and the residual delta is what reconciliation handles (positive → debt
allowed via `fail_insufficient_credits=False` from #13043; negative →
refund via USAGE row with negative cost).
## What
Replaces
`autogpt_platform/backend/backend/data/block_preflight_estimates.json`
with 9 dynamic-cost blocks aggregated from prod `CreditTransaction`
USAGE rows.
| Block | cost_type | samples (30d) | mean_credits |
|---|---|---:|---:|
| OrchestratorBlock | cost_usd | 3,227 | 6 |
| AITextGeneratorBlock | cost_usd | 2,283 | 4 |
| SearchTheWebBlock | cost_usd | 1,202 | 1 |
| AITextSummarizerBlock | cost_usd | 1,105 | 8 |
| ReplicateModelBlock | cost_usd | 633 | 10 |
| ExecuteCodeBlock | second | 63 | 1 |
| AIListGeneratorBlock | cost_usd | 36 | 5 |
| PerplexityBlock | cost_usd | 21 | 5 |
| AIStructuredResponseGeneratorBlock | cost_usd | 16 | 6 |
13 other dynamic-cost blocks on dev had <10 samples in the 30d window
and are intentionally omitted — they continue to pre-charge 0 (unchanged
from today). They'll get seeded once they have enough volume.
## How
1. Ran the same SQL as `compute_block_cost_estimates` (per-`(block_id,
node_exec_id)` sum, then mean across executions, `min_samples=10`, 30d
window) against the prod `CreditTransaction` table via `kubectl exec`
against an `autogpt-server` pod — no credentials left the cluster.
2. `cost_type` was re-resolved against `BLOCK_COSTS` on this branch to
match the dev runtime classification (no reclassifications needed — all
9 blocks have the same dynamic cost type on prod and dev).
3. Format matches the loader in `block_preflight_estimates.py` —
`version`, `generated_at`, `source_window_days` are metadata; only
`estimates.{block_id}.mean_credits` is consumed at runtime.
## Behaviour at runtime
For each of these 9 blocks, every execution now:
1. Pre-charges `mean_credits` at start (was 0).
2. Computes actual cost from execution stats post-flight.
3. Settles `delta = actual - mean_credits`:
- `delta > 0` → additional USAGE charge (wallet allowed to go negative).
- `delta < 0` → refund as USAGE row with negative cost.
- `delta == 0` → no-op.
**Net charge per execution is unchanged.** Only the timing of the debit
shifts forward.
## Test plan
- [ ] Spot-check OrchestratorBlock + AITextGeneratorBlock in PostHog /
Grafana — confirm the 30d mean is in the right ballpark.
- [ ] Verify `pnpm types` / `poetry run lint` pass (JSON-only change —
should be a no-op).
- [ ] Post-deploy on dev: run an OrchestratorBlock execution, confirm a
pre-flight USAGE row of ~6 credits, then a reconciliation row settling
the delta.
- [ ] No regression on TOKENS blocks (excluded by `DYNAMIC_COST_TYPES`;
`compute_token_credits` continues to provide the per-model floor at
pre-flight). Z
Zamil Majdy committed
e66b40b3698e83a19ec24a154d73c6231c7172f7
Parent: e4cc0b8
Committed by GitHub <noreply@github.com>
on 5/11/2026, 9:26:38 AM