feat(irodori-tts): add v4 support with shared pretrained text encoder
Irodori-TTS v4-Small replaces the two scratch-trained text/caption encoders with a single pretrained ModernBERT-ja-310m backbone feeding separate projectors, and raises speaker_patch_size to 4. - Port ModernBERT to MLX (modernbert.py): fused Wqkv, GeGLU MLP, alternating global/sliding-window attention with per-layer-type RoPE theta, and layer 0's identity attn_norm. Verified against transformers on random weights to 5e-7. - Add PretrainedTextBackbone + PretrainedConditionProjector and route text/caption encoding through them when text_encoder_type is "pretrained"; scratch encoders remain the default for v1-v3. - Load the tokenizer bundled with the converted weights when present, so inference no longer depends on the upstream tokenizer repo. - Support multi-clip reference audio: clips are encoded separately and concatenated, then trimmed to the checkpoint's ref_max_seconds (120s for v4, 30s before). - Size the no-reference placeholder latent to speaker_patch_size, which otherwise patches down to an empty sequence at v4's patch size 4. The sliding-window mask keeps its diagonal open so padding queries far from any real token cannot produce an all-masked softmax row. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Y
yoshphys committed
98a021d05aaa2084f37c004b09e35f400475bc57
Parent: 41aba81