Always tie embeddings for LongT5 and Pop2Piano (#47620)
* Always tie structural weights, regardless of `tie_word_embeddings`
`tie_word_embeddings` only controls whether the output embeddings are tied to
the input embeddings, but `get_expanded_tied_weights_keys` was returning an
empty mapping whenever it was False, dropping the structural entries as well.
For encoder-decoder models, `encoder.embed_tokens` and `decoder.embed_tokens`
are aliases of `shared` and are absent from the checkpoints, so they ended up
randomly initialized (e.g. longt5, which has `tie_word_embeddings=False`).
Only drop the entries pointing at `get_output_embeddings()` instead.
* Restore pre-#41580 tying expectations for fsmt/mbart
`tie_word_embeddings=False` only unties the output head; the encoder/decoder
embedding aliases are structural and stay tied. These two counts were 2 from
#26292/#26422 until #41580 bumped them to 3/4 to match the all-or-nothing gate.
* Add T5 tying test and document regex-form key handling
* Always tie embeddings for LongT5 and Pop2Piano
Reverts the framework-level change: `_tied_weights_keys` is an all-or-nothing
mapping gated by `tie_word_embeddings`, so `get_expanded_tied_weights_keys`
should not split it into "structural" and "head" entries. `modeling_utils` and
the fsmt/mbart/t5 test expectations are back to their state on main.
The actual bug is per-model. LongT5 and Pop2Piano are derived from T5 and
inherit its `tie_word_embeddings` quirk: the flag historically encoded "do not
scale the decoder outputs" (T5 1.1), not "do not share the embeddings". Every
official checkpoint sets it to `False` (`google/long-t5-local-base`,
`google/long-t5-tglobal-base`,
`Stancld/longt5-tglobal-large-16384-pubmed-3k_steps`, `sweetcocoa/pop2piano`)
while only storing `shared.weight`, so the aliased embeddings end up randomly
initialized.
Apply the same fix T5/MT5/UMT5 already have: force `tie_word_embeddings=True`
and move the scaling to its own `scale_decoder_outputs` flag.
Before, on `google/long-t5-local-base`:
encoder.embed_tokens.weight | MISSING
decoder.embed_tokens.weight | MISSING
lm_head.weight | MISSING
shared std 10.09 vs embed std 1.00
Pop2Piano is affected too, though only latently: the hub ships a `.bin`, which
preserves shared storage, but a safetensors round-trip stores `shared.weight`
alone and both `embed_tokens` come back randomly initialized.
* Shorten comments J
jiqing-feng committed
5909a467a38fa1bbdd12553dd358ac1a2486b33a
Parent: 94f09cf
Committed by GitHub <noreply@github.com>
on 8/20/2026, 10:01:15 AM