SIGN IN SIGN UP

fix(kimi_k25): map lm_head to model.language_model.lm_head (#3632)

fix(kimi_k25): map lm_head to model.language_model.lm_head, not a top-level key

KimiK25VLStateDictAdapter.from_hf renamed language_model.lm_head.* to
lm_head.*, which matches no parameter of KimiK25VLForConditionalGeneration
(the head lives at model.language_model.lm_head.*).

The base-model load runs set_model_state_dict(strict=False),
so the mismatch is only a warning
("missing=['model.language_model.lm_head.weight'] unexpected=['lm_head.weight']")
and the head keeps its random init: every K2.5 fine-tune through this path
trains against an uninitialised output head.

Apply the same "model." + key rule the adapter already uses for the other
language_model.* keys. With the fix the warning is gone and step-0 SFT loss
on chat data is 2.18 (a random head gives ~ln(163840) ≈ 12).

Signed-off-by: Aarni Koskela <akx@iki.fi>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
A
Aarni Koskela committed
9fb929703beece9be6efe45285a5dd77dba6b008
Parent: 9e3c455
Committed by GitHub <noreply@github.com> on 8/24/2026, 12:14:14 PM