SIGN IN SIGN UP

AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

0 0 73 Python

feat(backend): retire deprecated LLM models with family-aware migration (#13089)

### Why / What / How

**Why:** The `LlmModel` enum in `backend/blocks/llm.py` still contained
entries that providers have retired or superseded. Removing them was
unsafe because the boot-time safety net (`migrate_llm_models` in
`backend/data/graph.py`) force-mapped any out-of-enum value to
`DEFAULT_LLM_MODEL` (currently `GPT5_2`). That meant a Claude Opus user
would silently end up on GPT-5.2 — different provider, different price
tier, different behavior.

**What:** Retire 24 deprecated `LlmModel` members in one PR with:
1. a Prisma migration that rewrites stored model values (in both
`AgentNode.constantInput` and `AgentNodeExecutionInputOutput.data`) to
same-family replacements;
2. a `LEGACY_MODEL_MAPPINGS` dict that the safety net consults before
falling back to the global default;
3. removal of the dead entries from `MODEL_METADATA` / `MODEL_COST` /
`TOKEN_COST` and any pinned model strings in tests.

**How:**
- `LEGACY_MODEL_MAPPINGS: dict[str, LlmModel]` lives in
`backend/blocks/llm.py` as the single source of truth. The migration SQL
and the safety net both rewrite from the same mapping.
- `migrate_llm_models` now runs two SQL passes per LLM field: one
targeted `UPDATE` per legacy mapping (family-aware), then the existing
catch-all that maps anything still out-of-enum to `DEFAULT_LLM_MODEL`.
- AI/ML API stragglers (Qwen 2.5-72B, nvidia/llama-3.1-nemotron,
Meta-Llama-3.1-70B Turbo, Llama-3.2-3B Turbo) have no direct same-family
successor on AI/ML's current catalogue, so they all map to
`meta-llama/Llama-3.3-70B-Instruct-Turbo` (kept in the enum). This is
called out in the migration header.

Closes
[OPEN-3109](https://linear.app/autogpt/issue/OPEN-3109/retire-deprecated-llm-models-with-family-aware-migration).

### Changes 🏗️

- **New migration**
`migrations/20260512120000_retire_deprecated_llm_models/migration.sql` —
24 family-aware rewrites against `AgentNode.constantInput` and
`AgentNodeExecutionInputOutput.data` (preset overrides).
- **`backend/blocks/llm.py`** — removed 24 deprecated `LlmModel` members
+ their `MODEL_METADATA`; added `LEGACY_MODEL_MAPPINGS` shared with the
migration and the safety net.
- **`backend/data/graph.py`** — `migrate_llm_models` now does a targeted
`UPDATE` per legacy mapping before the catch-all fallback.
- **`backend/data/block_cost_config.py`** — dropped stale `MODEL_COST` +
`TOKEN_COST` entries for the removed models.
- **Tests** — updated pinned deprecated model strings in
`blocks/test/test_llm.py`, `data/credit_test.py`, `util/prompt_test.py`,
`copilot/config_test.py`, `copilot/model_normalize_test.py`, and 5
`copilot/sdk/*_test.py` fixtures.

#### Retired → replacement (verified against current provider docs)

| Retired | Replacement |
|--|--|
| `claude-3-haiku-20240307` | `claude-haiku-4-5-20251001` |
| `claude-opus-4-20250514` | `claude-opus-4-7` |
| `claude-sonnet-4-20250514` | `claude-sonnet-4-6` |
| `claude-opus-4-1-20250805` | `claude-opus-4-7` |
| `gpt-4-turbo` | `gpt-4.1-2025-04-14` |
| `o1` | `o3-2025-04-16` |
| `o1-mini` | `o3-mini` |
| `google/gemini-2.5-pro-preview-03-25` | `google/gemini-2.5-pro` |
| `google/gemini-2.5-flash-lite-preview-06-17` |
`google/gemini-2.5-flash` |
| `cohere/command-r-08-2024` | `cohere/command-a-03-2025` |
| `cohere/command-r-plus-08-2024` | `cohere/command-a-03-2025` |
| `mistralai/mistral-nemo` | `mistralai/mistral-small-3.2-24b-instruct`
|
| `microsoft/wizardlm-2-8x22b` | `microsoft/phi-4` |
| `moonshotai/kimi-k2` | `moonshotai/kimi-k2.6` |
| `moonshotai/kimi-k2-0905` | `moonshotai/kimi-k2.6` |
| `z-ai/glm-4-32b` | `z-ai/glm-4.6` |
| `z-ai/glm-4.5` | `z-ai/glm-4.6` |
| `z-ai/glm-4.5-air` | `z-ai/glm-4.7-flash` |
| `z-ai/glm-4.5-air:free` | `z-ai/glm-4.7-flash` |
| `z-ai/glm-4.5v` | `z-ai/glm-4.6v` |
| `Qwen/Qwen2.5-72B-Instruct-Turbo` |
`meta-llama/Llama-3.3-70B-Instruct-Turbo` |
| `nvidia/llama-3.1-nemotron-70b-instruct` |
`meta-llama/Llama-3.3-70B-Instruct-Turbo` |
| `meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo` |
`meta-llama/Llama-3.3-70B-Instruct-Turbo` |
| `meta-llama/Llama-3.2-3B-Instruct-Turbo` |
`meta-llama/Llama-3.3-70B-Instruct-Turbo` |

### Out of scope

- `DEFAULT_LLM_MODEL` is unchanged.
- Stagehand's `StagehandRecommendedLlmModel` subset references only
Claude 4.5/4.6 Sonnet (both kept) — untouched.
- No new models added. This PR is purely retirement + safety-net
hardening.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] `poetry run pytest
backend/data/graph_test.py::test_migrate_llm_models_uses_schema_prefix_placeholder`
— passes (regression: migrate_llm_models still uses the
`{schema_prefix}` placeholder for all queries, including the new
targeted ones).
- [x] `poetry run pytest backend/blocks/test/test_llm.py` — 60 tests,
all green.
- [x] Local `docker compose up -d` boot check — backend starts cleanly
and the boot-time safety net runs without errors (pending — please
verify in CI / before merge).
- [x] Apply migration against a staging snapshot of a database that
contains a graph pinned to one of the retired models, confirm
`constantInput.model` is rewritten to the mapped replacement (not
`DEFAULT_LLM_MODEL`).

#### For configuration changes:

- [x] `.env.default` is updated or already compatible with my changes —
no env changes.
- [x] `docker-compose.yml` is updated or already compatible with my
changes — no compose changes.
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**) — n/a.
K
Krzysztof Czerwinski committed
48129a9301bc48a32bfcf5c03097d9e863778a89
Parent: 270cd1e
Committed by GitHub <noreply@github.com> on 5/19/2026, 9:17:00 AM