fix(mllm-cache): block-align the media clean boundary so KV and the SSM companion can actually meet
Measured live on Qwen3.8-27B VL (M5 Max, SSD-only tier, block_size 64):
turn 2 stores: KV 7607 tokens, SSM companion at 7607
turn 3 fetches: Paged cache hit for ...: 117 blocks, checkpoint_tokens=7488
turn 3 looks for a companion at 7488 -> nothing, it is at 7607
VLM prefix cache MISS: 7488 KV blocks found but no SSM companion state
- full prefill required
The paged chain can only ever be MATCHED on block boundaries. A companion
stored at an unaligned N-1 is therefore invisible to every future turn, and
the fetch-side rescue cannot reach it either: fetch_longest_prefix only
returns checkpoints <= max_len, and the checkpoint sits ABOVE the hit. So a
found 7,488-token hit was discarded on turn 3, discarded again on turn 4, and
TTFT climbed 35s -> 60s -> 85s on a conversation that was only growing by
~3,700 tokens a turn.
The clean media boundary is now block-aligned before the capture runs, and
the companion is stored at that same length. Giving up at most block_size-1
tokens of stored prefix buys back the entire hit.
Keying: the aligned length travels on a NEW response field,
clean_store_token_ids, not on prompt_token_ids. The scheduler derives
request.num_prompt_tokens from len(prompt_token_ids) and that becomes the
user-visible usage.prompt_tokens -- shortening it would misreport every media
request. Both _extracted_tokens assignment sites prefer the new field, and a
test asserts BOTH are wired, since fixing one of two call sites is the
default failure mode in this file.
Tests: 5 new, including the exact 7607 -> 7552 -> 7488 arithmetic from the
live failure and a guard that the companion store and the capture use the
same length. J
Jinho Jang committed
524d9c3aa9679ef19f092227388f255837fbaf3a
Parent: 0c89045