Cpmant fix use cache (#48013)
* Fix CpmAnt loading: size lm_head to vocab_size The released openbmb/cpm-ant-10b checkpoint ships lm_head.weight with shape (vocab_size, hidden) = (30720, 4096), bit-identical to the first vocab_size rows of input_embedding.weight (31744, 4096). The trailing prompt_types * prompt_length rows are soft-prompt embeddings and are never decoding targets. The model built the head at vocab_size + prompt_types * prompt_length and declared a whole-tensor tie to the embedding, so from_pretrained hit a shape mismatch and then crashed in tie_weights comparing meta tensors. Size the head to vocab_size, which matches the checkpoint, and drop the whole-tensor tie mapping that could never have been valid at these shapes. * Fix CpmAnt cached generation CpmAntModel.forward prepends the soft prompt and rebuilds the attention mask and position bias over the whole sequence, then slices off the cached prefix itself, so it needs the full input_ids. #33870 removed the prepare_inputs_for_generation that supplied them, and the generic one slices down to the new tokens, so the position bias no longer lines up with the cache and generate raises RuntimeError: The size of tensor a (45) must match the size of tensor b (33) at non-singleton dimension 3 Restore the override, forwarding the full input_ids. * Shorten lm_head comment * Shorten prepare_inputs_for_generation comment
J
jiqing-feng committed
0f93813a69092a9afc9ee5375de64e7630e386f2
Parent: 51edaf8
Committed by GitHub <noreply@github.com>
on 8/26/2026, 6:10:22 AM