SIGN IN SIGN UP

Keep the hybrid APC parity harness bootable on 16 GB hosts (#639)

Fixes #638.

## Summary

The `--quick` gate (and the defaults arm in general) passed no
`max_num_batched_tokens`. `LLM()` therefore took the 8192 LLM-class
default,
and `profile_run()` evaluated a single `(1, 8192)` dummy. In the
reproduced
configuration on the tested 16 GB M-series Mac, the cache-on child's
profiled overhead reached 6.46 GB, its KV budget reached -1.81 GB, and
it
died during `LLM()` initialization. The cache-off child on the same
revision
initialized with 2.38 GB of overhead.

This change is limited to `tools/hybrid_apc_parity_matrix.py`:

- The defaults arm now passes `max_num_batched_tokens=2048`, matching
its
existing `max_model_len`. The arm no longer exercises the LLM-class 8192
default. That default was not needed by the short parity prompts; it
made
  the profile dummy four times longer than any legal sequence in this
  harness. The explicit `mnbt1088` arm is unchanged.
- The parent polls the queue in five-second intervals and fails
immediately
  with the child exit code if the child terminates before `queue.put()`,
  instead of waiting up to 1500 seconds and then raising `queue.Empty`.

This PR changes no serving behavior or library default. A serving-shaped
profile path is separate work being discussed in RFC #525 and #590.
#585/#586 concerned the former MLX per-buffer default; #637's proposed
conditional default does not apply below 50 GiB usable memory and is not
involved on the tested host.

## Testing

- `python tools/hybrid_apc_parity_matrix.py --quick` — 20 comparisons,
  0 mismatches, `PARITY PASS`
- Cache-on profile: overhead 6.46 GB -> 0.73 GB; KV budget -1.81 GB ->
  +3.92 GB
- Cache-off bounded profile: overhead 0.76 GB; final KV budget +3.83 GB

The end-to-end wrapper is marked `slow`, is not run in CI, and requires
local Qwen/Qwen3.5-0.8B weights.

## AI assistance

AI assistance was used in developing this change. The problem analysis,
the patch, and the measurements were reviewed and verified by me, and I
am responsible for all submitted changes.

Signed-off-by: Polyphonia.it <noreply@polyphonia.it>
Co-authored-by: Polyphonia.it <noreply@polyphonia.it>
F
fdaluiso committed
6ef8313ec94c821f7bea9be6bb3e7198fc899329
Parent: 41f3ed3
Committed by GitHub <noreply@github.com> on 8/25/2026, 3:48:06 AM