SIGN IN SIGN UP

Always tie embeddings for LongT5 and Pop2Piano (#47620)

* Always tie structural weights, regardless of `tie_word_embeddings`

`tie_word_embeddings` only controls whether the output embeddings are tied to
the input embeddings, but `get_expanded_tied_weights_keys` was returning an
empty mapping whenever it was False, dropping the structural entries as well.

For encoder-decoder models, `encoder.embed_tokens` and `decoder.embed_tokens`
are aliases of `shared` and are absent from the checkpoints, so they ended up
randomly initialized (e.g. longt5, which has `tie_word_embeddings=False`).

Only drop the entries pointing at `get_output_embeddings()` instead.

* Restore pre-#41580 tying expectations for fsmt/mbart

`tie_word_embeddings=False` only unties the output head; the encoder/decoder
embedding aliases are structural and stay tied. These two counts were 2 from
#26292/#26422 until #41580 bumped them to 3/4 to match the all-or-nothing gate.

* Add T5 tying test and document regex-form key handling

* Always tie embeddings for LongT5 and Pop2Piano

Reverts the framework-level change: `_tied_weights_keys` is an all-or-nothing
mapping gated by `tie_word_embeddings`, so `get_expanded_tied_weights_keys`
should not split it into "structural" and "head" entries. `modeling_utils` and
the fsmt/mbart/t5 test expectations are back to their state on main.

The actual bug is per-model. LongT5 and Pop2Piano are derived from T5 and
inherit its `tie_word_embeddings` quirk: the flag historically encoded "do not
scale the decoder outputs" (T5 1.1), not "do not share the embeddings". Every
official checkpoint sets it to `False` (`google/long-t5-local-base`,
`google/long-t5-tglobal-base`,
`Stancld/longt5-tglobal-large-16384-pubmed-3k_steps`, `sweetcocoa/pop2piano`)
while only storing `shared.weight`, so the aliased embeddings end up randomly
initialized.

Apply the same fix T5/MT5/UMT5 already have: force `tie_word_embeddings=True`
and move the scaling to its own `scale_decoder_outputs` flag.

Before, on `google/long-t5-local-base`:

    encoder.embed_tokens.weight | MISSING
    decoder.embed_tokens.weight | MISSING
    lm_head.weight              | MISSING
    shared std 10.09 vs embed std 1.00

Pop2Piano is affected too, though only latently: the hub ships a `.bin`, which
preserves shared storage, but a safetensors round-trip stores `shared.weight`
alone and both `embed_tokens` come back randomly initialized.

* Shorten comments
J
jiqing-feng committed
5909a467a38fa1bbdd12553dd358ac1a2486b33a
Parent: 94f09cf
Committed by GitHub <noreply@github.com> on 8/20/2026, 10:01:15 AM