fix(backend): centralise paywall enforcement on all execute routes (NO_TIER bypass) (#13045)

## Why

Live testing on prod surfaced this regression: a NO_TIER user
(post-cutoff signup) sees the frontend `PaywallModal` correctly, but
**autopilot turns + agent execution + block runs all still execute on
the backend**. The paywall is purely cosmetic.

Toran reproduced on prod with `toran-tester-2@autogpt.ai` and
`toran-tester-3@autogpt.ai` (both NO_TIER, both with
`enable-platform-payment: True`).

## What

Two semantics conflicts in
[`copilot/rate_limit.py`](autogpt_platform/backend/backend/copilot/rate_limit.py)
plus a missing function-level gate:

1. `_DEFAULT_TIER_MULTIPLIERS[NO_TIER] = 0.0` was documented as "the
backend half of the paywall — collapses limits to zero."
2. `get_global_rate_limits` applied the multiplier and returned
`(daily=0, weekly=0, tier=NO_TIER)`.
3. **`check_rate_limit` then short-circuited**: `if daily_cost_limit <=
0 and weekly_cost_limit <= 0: return` — interpreting `(0, 0)` as
"unlimited" rather than "blocked".

Beyond the chat path, `add_graph_execution` (the central enqueue used by
`/v1/graphs/{id}/execute`, scheduled cron, webhook triggers, copilot
internal tools, and the external API) had **no tier check at all** —
only a `credits <= 0` gate, which onboarding-granted credits trivially
satisfied.

## How

**Architecture: 2-function paywall API, deep gate, app-level handler.**

- **`is_user_paywalled(user_id) -> bool`** in `rate_limit.py` is THE
definition of "paywalled". Treats `_UserNotFoundError` as NO_TIER (fresh
signup case). Lookup errors propagate — callers decide their own
failure-mode posture.
- **`enforce_payment_paywall(user_id)`** is THE HTTP gate. Wraps
`is_user_paywalled` and raises `UserPaywalledError` on paywall (handler
→ 402) or `HTTPException(503)` + Retry-After on lookup error. Doubles as
JWT route dep (FastAPI fills `user_id` via `Security`) AND inline call
from API-key routes (explicit positional `user_id` overrides the
`Security` default).
- **`UserPaywalledError`** is THE exception.
- **App-level `@exception_handler(UserPaywalledError)`** maps to HTTP
402 once (registered in both `rest_api.py` and the external API's
`fastapi_app.py`).

**Two call shapes, same primitive:**

1. **HTTP routes (JWT + API-key)**: `enforce_payment_paywall` — as a
`Depends(...)` for JWT, or inline `await
enforce_payment_paywall(auth.user_id)` for API-key.
2. **Background callers** (`add_graph_execution` deep gate, copilot
internal tools, scheduled cron, webhook handlers): use
`is_user_paywalled` directly + `raise UserPaywalledError(...)` inline.
They skip the HTTP wrapper because synthesising an `HTTPException` is
the wrong shape outside an HTTP context — the framework's own retry
layer handles lookup errors instead.

**`check_rate_limit` cleanup** (kills the underlying ambiguity):
- Removed the `<= 0 means unlimited` short-circuit (legacy convention,
no real-world tier behind it).
- Changed `> 0` → `>= 0` so a limit of 0 means "no spend allowed", not
"unlimited".
- Aligned `get_remaining_usd_budget`, `build_budget_ctx`, `from_status`,
and all docstrings to the same semantics.

**Admin recovery escape hatch**: `add_graph_execution` accepts
`bypass_paywall: bool = False` (keyword-only). The three admin requeue
endpoints (`requeue_single_execution`, `requeue_multiple_executions`,
`requeue_all_stuck_executions`) pass `bypass_paywall=True` so admins can
recover stuck executions for users who have since downgraded to NO_TIER.
Set ONLY by admin routes — never for user-initiated runs.

**Self-hosters / open-source unaffected**: without LD configured,
`is_feature_enabled` returns False → `is_user_paywalled` returns False →
paywall doesn't fire. `get_global_rate_limits` for NO_TIER falls back to
BASIC's multiplier (1.0) so usage caps stay positive and
`check_rate_limit` passes normally.

## Coverage table

| Entry point | Gate | Paid tier | NO_TIER + flag-off (beta) | NO_TIER +
flag-on | DB-down |
|---|---|---|---|---|---|
| `POST /v1/chat/sessions/{id}/stream` | route dep
`enforce_payment_paywall` | ✅ pass | ✅ pass (BASIC fallback) | ❌ **402**
| ❌ **503** + Retry-After |
| `POST /v1/blocks/{id}/execute` | route dep `enforce_payment_paywall` |
✅ pass | ✅ pass | ❌ **402** | ❌ **503** |
| `POST /external/v1/blocks/{id}/execute` | inline
`enforce_payment_paywall(auth.user_id)` | ✅ pass | ✅ pass | ❌ **402**
(via app handler) | ❌ **503** + Retry-After |
| `POST /v1/graphs/{id}/execute/{ver}` | route dep
`enforce_payment_paywall` + deep gate | ✅ pass | ✅ pass | ❌ **402** | ❌
**503** + Retry-After |
| `POST /external/v1/graphs/{id}/execute/{ver}` | inline
`enforce_payment_paywall(auth.user_id)` + deep gate | ✅ pass | ✅ pass |
❌ **402** | ❌ **503** + Retry-After |
| Webhook trigger → graph enqueue | deep gate (`is_user_paywalled`
inline) | ✅ pass | ✅ pass | ❌ raises (event marked failed) | ❌ raises
(caller's retry layer) |
| Scheduled cron → graph enqueue | deep gate (`is_user_paywalled`
inline) | ✅ pass | ✅ pass | ❌ raises (job marked failed) | ❌ raises
(caller's retry layer) |
| Copilot internal tool → graph enqueue | deep gate (`is_user_paywalled`
inline) | ✅ pass | ✅ pass | ❌ raises | ❌ raises (caller's retry layer) |
| Admin: `POST
/api/admin/diagnostics/executions/requeue{,-bulk,-all-stuck}` |
`bypass_paywall=True` — admin recovery, gate skipped | ✅ | ✅ | ✅ (admin
recovery allowed) | ✅ (admin recovery allowed) |
| `POST /v1/graphs/.../schedule` (creates schedule, doesn't execute) |
none — schedule creation allowed; execution gated when it fires | ✅ | ✅
| ✅ pass; runtime gate on execution | n/a |

**Adding a new paywalled entry point:**
- If it calls `add_graph_execution` → automatic, no code needed.
- If it's a JWT route → add
`dependencies=[Depends(enforce_payment_paywall)]`.
- If it's API-key auth or other non-JWT → `await
enforce_payment_paywall(user_id)` inline.
- If it's an admin recovery path on someone else's execution → call
`add_graph_execution(..., bypass_paywall=True)`.

## Caching

- `_fetch_user_tier` — `@cached(ttl_seconds=300, maxsize=10000,
shared_cache=True)` — 5-min TTL on tier lookups, Redis-backed across
pods. `_UserNotFoundError` is NOT cached, so a fresh signup sees their
tier flip to provisioned within one Prisma round-trip rather than
waiting up to 5 minutes.
- `_fetch_user_context_data` (LD context) — `@cached(ttl_seconds=86400,
maxsize=1000)` — 24-hour TTL
- `_fetch_tier_multipliers_flag` / `_fetch_cost_limits_flag` —
`@cached(ttl_seconds=60)`
- `is_user_paywalled` itself isn't cached but its primitives are — a hot
user pays one fast in-memory call.

## Test plan

- [x] `TestCheckRateLimit` — 14 tests including the headline
`test_zero_limit_means_blocked_not_unlimited`
- [x] `TestEnforcePaymentPaywall` + `TestEnforcePaymentPaywallContinued`
+ `TestEnforcePaymentPaywallInline` — NO_TIER+on→`UserPaywalledError`,
NO_TIER+off→pass, all paid tiers parametrized→pass, DB failure→503
- [x] `TestIsUserPaywalled` — 8 tests including `_UserNotFoundError →
NO_TIER` collapse for fresh signups
- [x] `TestFetchUserTierErrorPropagation` — `ValueError →
_UserNotFoundError → NO_TIER` vs other exceptions propagate (so DB
outages don't silently 402 paying users)
- [x] `TestUserPaywalledError` — 2 tests
- [x] `TestCoPilotUsagePublicFromStatus` — 2 tests (limit=0 surfaces as
100% used, not null/unlimited)
- [x]
`TestGetGlobalRateLimitsWithTiers::test_no_tier_with_payment_disabled_upgrades_to_basic`
— beta cohort regression
- [x] `TestGetRemainingUsdBudget` + `TestBuildBudgetCtx` — updated to "0
= no remaining" semantics
- [x] `add_graph_execution` paywall-gate coverage in
`executor/utils_test.py`: paywalled-user-blocked +
`bypass_paywall=True`-skips-gate
- [x] Autouse `_bypass_paywall` fixture in `backend/api/conftest.py` AND
`backend/executor/conftest.py` — every test treats user as paid by
default; paywall-specific tests opt out by patching `_fetch_user_tier`
- [x] External API graph-execute route adds `except UserPaywalledError:
raise` to bypass its broad `except Exception → 400` (otherwise the
catch-all would swallow paywall into 400)
- [x] `openapi.json` regenerated for the 402 / 503 declarations
- [x] **189 rate_limit + executor tests pass** locally; full suite green
pre-push.
- [x] Live e2e smoke — PASS, see [results
comment](https://github.com/Significant-Gravitas/AutoGPT/pull/13045#issuecomment-4402686777).
All 5 routes return 402 for NO_TIER+flag-on and pass cleanly for the
other 2 scenarios.

## Reviewer notes

- **ntindle** (route-level dep pattern matching `requires_admin_user`) →
adopted; extended with the function-level gate to cover non-route entry
points.
- **sentry-bot** (the `>= 0` regression for `0=unlimited` configs;
`from_status` projecting 0 → null; `_UserNotFoundError` → 500 on
external block route; admin requeue blocked for NO_TIER users by the new
gate) → all addressed; "0 = unlimited" convention removed everywhere;
0-limit windows surface as 100%-used in public projection; missing-user
collapses to NO_TIER inside `is_user_paywalled`; `bypass_paywall=True`
on admin requeue paths.
- **CodeRabbit** (missing `NO_TIER + flag-off → BASIC fallback`
regression test; `get_user_tier`'s fail-open default leaking into the
dep; broad `except Exception` swallowing `UserPaywalledError` in
external graph route; `_fetch_user_tier` masking transient DB outages as
missing-user) → all addressed; tests added; route uses
`_fetch_user_tier` directly with explicit 503 mapping; surgical re-raise
added; `_fetch_user_tier` now catches only `ValueError` (the documented
"user not found" signal) so transient errors propagate to the 503
mapper.