fix: split large embedding batches that exceed provider aggregate token limits (#19565)
## Background Large OpenAI and Azure OpenAI embedding batches failed with HTTP 400 instead of returning all embeddings because requests could exceed the providers' 300,000-token aggregate limit. ## Root Cause `embedMany` split requests only by the model's input-count limit. The reproduction confirmed this produced 2,048- and 130-value requests, with the first exceeding the aggregate token limit. ## Summary - Adds an experimental symbol capability in `@ai-sdk/provider-utils` for embedding-model UTF-8 input byte budgets, leaving the versioned `EmbeddingModelV4` specification unchanged. - Configures conservative 300,000-byte budgets for OpenAI and Azure OpenAI. - Splits embedding inputs in one pass by both input count and byte budget while preserving per-chunk retries, telemetry, and response metadata. - Propagates the experimental capability through embedding middleware wrappers. - Documents the batching behavior and includes patch changesets. ## Testing Added regression coverage for byte-only splitting, exact boundaries, combined limits, unlimited budgets, oversized single inputs, middleware capability propagation, and the OpenAI and Azure OpenAI configured budgets. ## End-to-end Validation - `pnpm -C examples/ai-functions exec tsx src/reproduction/issue-10082-embed-many-token-limit.ts` succeeded against OpenAI, returning all 2,178 embeddings without the aggregate-token-limit error. ## Related Issues Fixes #10082 Closes #17073 --------- Co-authored-by: ai-sdk-factory <308175966+ai-sdk-factory@users.noreply.github.com> Co-authored-by: Gregor Martynus <39992+gr2m@users.noreply.github.com> Co-authored-by: eposha <54452472+eposha@users.noreply.github.com>
A
ai-sdk-factory[bot] committed
d2f335310d41beadb32205434622d8680db39c39
Parent: 0f25f5d
Committed by GitHub <noreply@github.com>
on 8/25/2026, 10:00:54 PM