AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.
fix(backend): unblock orchestrator dry-run end-to-end (canonical model + SDK auth) (#13180)
### Why / What / How
**Why:** Two follow-ups landed after PR #13177 deployed and the user
exercised the orchestrator dry-run on dev-builder.
1. **JSON Schema validation gap.** #13177 taught `LlmModel._missing_` to
resolve `anthropic/claude-haiku-4-5` → `CLAUDE_4_5_HAIKU`. That fixed
Pydantic, but `OrchestratorBlock.Input` is validated by `validate_data`
→ `jsonschema.validate` *first*, against a schema whose `enum` is the
literal list of `LlmModel.value` strings. The alias map is a Python
runtime hook — it does not surface in the generated JSON Schema. So the
OR-slug was rejected with `'anthropic/claude-haiku-4-5' is not one of
[...]`.
2. **SDK auth: empty `x-api-key`.** Once jsonschema started passing,
execution reached the EXTENDED_THINKING SDK path in
[orchestrator.py:1670-1674](autogpt_platform/backend/backend/blocks/orchestrator.py#L1670).
That code sets `sdk_env["ANTHROPIC_API_KEY"] = ""` to "force the CLI to
use AUTH_TOKEN." But the Claude Agent SDK merges `options.env` on top of
`os.environ`
([subprocess_cli.py:402](file:claude_agent_sdk/_internal/transport/subprocess_cli.py#L402)),
so the spawned CLI sees `ANTHROPIC_API_KEY=` (present-but-empty) and
emits `x-api-key:` (empty header) on the wire. OpenRouter returns `401
invalid x-api-key`. This bug was latent until #13177 — every
orchestrator dry-run died at jsonschema before reaching the auth wiring.
**What:**
- `simulator.py`: translate the configured simulator model to its
canonical `LlmModel.value` via `LlmModel(_simulator_model()).value`
before injecting into `input["model"]`. The OR-slug default
(`anthropic/claude-haiku-4-5`) becomes `claude-haiku-4-5-20251001` —
which IS in the JSON Schema enum — so `validate_data` passes. Downstream
OpenRouter's Anthropic-compat endpoint accepts both forms, so no further
translation is needed.
- `orchestrator.py`: in the `provider == "open_router"` SDK branch, set
`ANTHROPIC_API_KEY` to the same OpenRouter key (instead of `""`).
OpenRouter accepts either `x-api-key` or `Authorization: Bearer` with
the OR key, so whichever the CLI sends is valid. Explicit set (not
omission) is required because the SDK's merge would otherwise let an
inherited platform `ANTHROPIC_API_KEY` leak through.
- `simulator_test.py`: strengthen
`test_orchestrator_uses_simulation_model` to assert the injected model
is in `{m.value for m in LlmModel}`; add
`test_orchestrator_input_passes_jsonschema_validation` that calls
`validate_data` on `prepare_dry_run`'s output (locks in the exact
regression the user hit).
**How:**
End-to-end proof for the model translation, against real OpenRouter
(4-step script):
```
=== Step 1: reproduce the bug with the OR slug ===
✓ validate_data REJECTS 'anthropic/claude-haiku-4-5' as expected
'anthropic/claude-haiku-4-5' is not one of ['o3-mini', ...]…
=== Step 2: canonical snapshot passes validate_data + Pydantic ===
✓ validate_data accepts the canonical value
✓ Pydantic resolves Input.model to LlmModel.CLAUDE_4_5_HAIKU
=== Step 3: prepare_dry_run shape after canonical-model fix ===
prepare_dry_run produced model='anthropic/claude-haiku-4-5'
Canonical-translated model='claude-haiku-4-5-20251001'
✓ dry-run input (post-fix) passes validate_data
=== Step 4: real LLM call via OpenRouter Anthropic-compat ===
✓ Real LLM call succeeded with canonical snapshot ID
```
The SDK auth fix is supported by the user-reported 401 + the SDK-source
trace
([subprocess_cli.py:402](file:claude_agent_sdk/_internal/transport/subprocess_cli.py#L402)
merges options.env on top of os.environ; setting "" leaves the key set
rather than unset).
### Changes 🏗️
- `prepare_dry_run` translates the simulator model to canonical
`LlmModel.value`.
- Orchestrator's SDK env now uses the OR key for both
`ANTHROPIC_AUTH_TOKEN` and `ANTHROPIC_API_KEY` (was empty string).
- Two strengthened/new tests pin the canonical-value invariant and
jsonschema validation outcome.
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] `poetry run pytest
backend/executor/simulator_test.py::TestPrepareDryRun
backend/executor/simulator_test.py::TestDefaultSimulatorModel
backend/copilot/tools/test_dry_run.py::test_prepare_dry_run_orchestrator_block
backend/blocks/test/test_llm.py::TestLlmModelMissingHandler` — all green
- [x] End-to-end validation against real OpenRouter — all 4 steps pass
(see "How")
- [x] User-reported 401 on dev-builder traced to the orchestrator SDK
env-var; root cause + SDK-source line cited
- [x] `poetry run black` + `poetry run ruff check` clean on changed
files
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Z
Zamil Majdy committed
6a6f2d2c23afbac0aaf46f17871695fd4a2e277c
Parent: 39f69e7
Committed by GitHub <noreply@github.com>
on 5/21/2026, 4:37:13 PM