fix: vLLM provider should respect user-configured contextLength and model (#11868)
* fix: respect user-configured maxTokens and contextLength for autocomplete Two fixes for autocomplete's maxTokens not being respected (#6003): 1. vLLM provider: _setupCompletionOptions() unconditionally overwrote the user's contextLength and model with server-reported values. Now user-explicit settings take priority over auto-detected values. 2. Autocomplete pipeline: renderPromptWithTokenLimit() returned completionOptions without maxTokens, relying on an implicit merge deeper in the call stack. Now maxTokens is explicitly included in the returned completionOptions so the value flows through to the API request reliably. * remove verbose comments from Vllm provider * style: fix prettier formatting * fix: only set maxTokens when llm provides a value * remove defensive-only maxTokens pass-through from autocomplete pipeline
D
Dallin Romney committed
f7907fff08719bd9847f1eb65c6967c302768f4a
Parent: 393c925
Committed by GitHub <noreply@github.com>
on 3/26/2026, 5:21:11 AM