SIGN IN SIGN UP

Preserve trailing whitespace when buffering local LLM stream remainder

TextIteratorStreamer chunks from the local transformers and MLX backends
end at word boundaries with a trailing space. In _process_printable_text,
nltk's sent_tokenize strips that space from the kept remainder
(sentences[-1]), so the next streamed chunk is appended directly onto the
truncated word: buffered "It is sunny. What is " plus "the weather
like?" becomes "What isthe weather like?", which is batched and sent to
TTS as a fused word.

PR #481 (merged, closes #480) applied exactly this trailing_whitespace
fix to base_openai_compatible_language_model.py, but only there; the
local-model sibling in language_model.py was never touched. PR #497
(merged) patched both files in one change, treating them as siblings.
This mirrors #481: capture the trailing whitespace before tokenizing and
re-append it to the kept remainder.

Adds a regression test driving _stream_tokens with split deltas in the
style of #481's test_audio_streaming_preserves_provider_whitespace_across_chunks.
J
Jeremy Schoemaker committed
056f1f390e957a9ec0060bf404628443ce416853
Parent: 64e2af0