SIGN IN SIGN UP

📄 Proposal: send PDFs as file_data (#89) (#140)

## Change Log / Overview

📄 Proposed PDF pass-through for #89. `editTools` stays override-only
after review: a non-empty hint short-circuits Copilot Chat's own
edit-tool preferences, so derived capabilities no longer guess a list.

### 1. PDF data parts reach LiteLLM (`file.file_data`)

Both message converters silently dropped `application/pdf` data parts,
even though the token estimators already counted them and the gateway
advertises `supports_pdf_input: true` for Claude, Gemini, and Azure GPT
routes.

The issue text suggested reusing the image shape. I probed that against
a real one-page PDF on `/chat/completions` and it doesn't hold up:

| Content shape | Vertex Gemini | Bedrock/Vertex Claude | Azure GPT |

|-------------------------------------------------------|---------------|-----------------------|---------------|
| `image_url` with `data:application/pdf;base64,…` | reads PDF | reads
PDF | **400** |
| `file.file_data` with `data:application/pdf;base64,…` | reads PDF |
reads PDF | **reads PDF** |
| `file.file_data` with bare base64 (no `data:` prefix) | 400 | 500 |
400 |
| `input_file` | 500 | 500 | 500 |
| Anthropic-native `document.source.base64` | 200, ignored | reads PDF |
400 |

Azure is explicit: `Invalid image URL … Expected a base64-encoded data
URL with an image MIME type … but got unsupported MIME type
'application/pdf'`.

So PDFs use `{ type: "file", file: { filename, file_data } }` and images
keep `image_url`. `file.file_data` is the only shape all three families
accept, which is why it's applied unconditionally rather than branched
per provider.

### 2. Removed duplicated capability mapping

`buildCapabilities()` in the provider base was unused outside its own
tests, and the commit provider hardcoded `{ toolCalling: true,
imageInput: false }`. Both now go through the shared registry
derivation. `editTools` is still only emitted when
`modelCapabilitiesOverrides` lists recognized tool ids.

---

### ⚠️ What is **not** verified

VS Code has not been observed handing this provider an `application/pdf`
`LanguageModelDataPart` at all. Copilot Chat resolves attached PDFs with
workspace/terminal tools first (`read_file`, `cat`, even `pdfplumber`),
and `LanguageModelChatCapabilities` has no PDF/document capability a
BYOK provider could advertise — so the host has no way to know the model
would accept one. Tested on WSL remote and Windows local, drag-drop and
attach.

The conversion path is therefore verified by running the **compiled
converter's own output** against the gateway rather than through the
chat UI:

```
gpt-5.6-luna                       HTTP 200 ok=true :: ORANGE-MANGO-PROTOCOL-42
vertex_ai/gemini-3.1-flash-lite    HTTP 200 ok=true :: ORANGE-MANGO-PROTOCOL-42
claude-haiku-4-5                   HTTP 200 ok=true :: ORANGE-MANGO-PROTOCOL-42
```

`/responses`-routed PDFs are forwarded by the adapter and covered by a
unit test, but have not been live-probed.

So the PDF change is best read as removing a guaranteed Azure 400 the
moment a PDF part ever does arrive, rather than as a user-visible fix
today.

Copilot Chat's PDF attach path still gates on `supportsVision` plus a
hardcoded `family` allowlist. That is tracked upstream as
microsoft/vscode#324961; this PR does not attempt to spoof `family`.

## Related Issues, Builds, Pipeline Runs, etc.

- #89
- microsoft/vscode#324961

## Pull Request Pre-check

- [x] Linting Validation Passed — 0 errors (31 pre-existing warnings,
none in touched files)
- [x] Formatting Validation Passed
- [x] Unit Testing Passes — 1047 passing; S 91.31 / B 82.60 / F 89.17 /
L 91.31
- [ ] Documentation Updated — no user-facing docs changed; `editTools`
override syntax in `README.md` is unchanged
D
David Tai committed
7032bd85a080662ef24b39fb7a12c03347747d5f
Parent: 5da2e21
Committed by GitHub <noreply@github.com> on 8/28/2026, 1:12:17 PM