AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.
fix(backend/copilot): mandate find_block before refusal + suppress false empty-completion on retry (#13086)
## Why
Two production-visible regressions surfaced in the same dev session
(`d077d025-f6a3-455f-836f-9fe3039b2b98`):
1. **AutoPilot refused without searching.** User asked "Post a test post
to
linkedin for me". The model emitted *"We don't have a native LinkedIn
integration yet. LinkedIn's API is also heavily restricted…"* with
**zero
tool calls** — never invoked `find_block`. When the user pushed back,
the
model immediately found `PostToLinkedInBlock` (Ayrshare) on the first
`find_block(query="linkedin post")`. The model's own thinking from
langfuse: *"The user is correct — I jumped straight to capability gap
without first searching for blocks. Per my instructions, I should always
`find_block` first for any integration request."* — i.e. the rule
existed
but was overridden by a vivid worked example in the system prompt.
2. **"Model returned an empty response" overlay on top of working
output.**
In the same session's recovery turn, after `find_block` + `run_block`
succeeded and the model emitted *"Yes, we have a `PostToLinkedInBlock`
via Ayrshare — and you're already connected…"*, a retry-adapter
recreated mid-turn lost the prior attempt's `has_started_text` /
`_any_tool_results_seen` state and false-fired
`code=empty_completion` on the next ResultMessage.
Root causes:
- The Langfuse "CoPilot Prompt" Section 4 ("Capability Check") + Example
3
(Shopify) explicitly modelled "We don't have a native [Platform]
integration yet" as the *correct* response when the model "knows" the
platform isn't supported — training the model to refuse without
searching. The shared tool-notes supplement said "find_block first" but
the worked example won.
- `SDKResponseAdapter` is rebuilt fresh on context-reduce retries
(service.py:4510). Its empty-completion guard relies on per-attempt
state, so a benign empty trailing ResultMessage on a retry — when the
prior attempt already streamed text/tools to the wire — is
indistinguishable from a true SECRT-2252 ghost finish and surfaces a
spurious error overlay.
## What
Code-side fixes (Langfuse prompt is also patched on the `latest`/dev
label out-of-band; prod stays on its existing version pending review):
- **`prompting.py`** — rewrite `SHARED_TOOL_NOTES` "Tool Discovery
Priority" to mark `find_block` as MANDATORY before any "no integration"
reply. Add an explicit "Anti-pattern: refusing without searching"
section that enumerates the exact phrases (`We don't have a native X
integration yet.`, `There's no block for X.`, etc.) and shows the
correct 3-step flow.
- **`response_adapter.py`** — add
`prior_attempt_emitted_visible_content: bool = False` to
`SDKResponseAdapter`; `_is_empty_completion` returns False when the
flag is set, so the SECRT-2252 guard only fires when the *broader
stream* has produced no content, not just the current attempt.
- **`service.py`** retry path — before swapping the adapter on
`_reduce_context` retry, set the new adapter's
`prior_attempt_emitted_visible_content` based on the prior adapter's
`has_started_text` / `has_started_reasoning` /
`_any_tool_results_seen` (and forward the flag transitively across
multi-retry chains).
- **Tests** — `prompting_test.py::TestToolDiscoveryPriorityAntiPattern`
(3 assertions on the new mandatory language + anti-pattern phrases +
correct-flow template); `response_adapter_test.py` adds
`test_retry_adapter_with_prior_emitted_content_suppresses_empty_completion`
(positive: empty success on retry → StreamFinish, no StreamError) and
`test_retry_adapter_without_prior_content_still_surfaces_empty_completion`
(counter-test: SECRT-2252 still fires when the flag is False).
## How
The three fixes layer:
1. **System-prompt rule** (defence in depth) — code-side supplement now
explicitly forbids the failure-mode phrases by exact wording; the
model can pattern-match its own draft and reject it.
2. **Langfuse worked example** (root cause) — Section 4 + Example 3 of
"CoPilot Prompt" rewritten on `latest` v30 to model search-first
behaviour. Prod (`production` v27) untouched until reviewed.
3. **Adapter false-positive guard** — surface "model returned an empty
response" only when the *whole stream* produced nothing, not just the
current retry attempt.
## Test plan
- [x] `pytest backend/copilot/sdk/response_adapter_test.py -k
"empty_completion or prior_attempt or retry_adapter"` — 3 passed.
- [x] `pytest
backend/copilot/sdk/response_adapter_test.py::test_retry_adapter_with_prior_emitted_content_suppresses_empty_completion
backend/copilot/sdk/response_adapter_test.py::test_retry_adapter_without_prior_content_still_surfaces_empty_completion`
— 2 passed.
- [x] Direct supplement assertion (`from backend.copilot.prompting
import get_sdk_supplement; ...`) — all pass.
- [x] `ruff check` clean on edited files.
- [x] Validated against trace `9f3feb3dd843aa1e537de92f23666cc9` /
`746b49e0f6012ff94035d0ceb3f7fca6` (the LinkedIn dev session).
- [ ] Manual dev verification with a fresh integration request after
`latest` v30 propagates (≤5 min Langfuse cache TTL). Z
Zamil Majdy committed
8d9ecb8197c39ecf61b760cd60f930099d7b294a
Parent: 12cc0f6
Committed by GitHub <noreply@github.com>
on 5/12/2026, 9:49:45 AM