SIGN IN SIGN UP

Support optimized multifunction SpeechDecoder for Qwen3-TTS (#494)

TTSKit's Qwen3-TTS SpeechDecoder now uses an optimized, multifunction Core ML asset that bundles two graphs in one bundle and lets you trade time-to-first-audio against throughput:

| Mode | Frames / call | Audio / call | Best for |
|---|---|---|---|
| `.latencyOptimized` (default) | 1 | ~80 ms | Lowest TTFB — streaming UX |
| `.throughputOptimized` | 4 | ~320 ms | Faster overall on longer clips |

The default `.latencyOptimized` path produces the same audio as before, just faster. `.throughputOptimized` batches four frames per call to amortize decoder cost, at the cost of a larger first buffer. The new asset ships on the public [`argmaxinc/ttskit-coreml`](https://huggingface.co/argmaxinc/ttskit-coreml) repo and is picked up by default — no token or repo override needed.

## What's new

- **`Qwen3SpeechDecoderMode`** (`.latencyOptimized` / `.throughputOptimized`), selectable via `TTSKitConfig.speechDecoderMode` (default `.latencyOptimized`).
- **CLI:** `argmax-cli tts --speech-decoder-mode latencyOptimized|throughputOptimized`.
- **Example app:** a Latency / Throughput picker in the sidebar (flipping it reloads the model), plus a waveform fix so throughput-mode playback renders at the right length.
- **Internals:** `KVCache` / `SpeechDecoderCache` are generalized to handle the asset's batched I/O; the number of frames per call is read from the loaded model, so future re-tunings need no code changes.

## Usage

```swift
// Default: latency-optimized (lowest time-to-first-audio)
let tts = try await TTSKit()

// Opt into throughput-optimized generation
let config = TTSKitConfig(speechDecoderMode: .throughputOptimized)
let throughputTTS = try await TTSKit(config)
```

```bash
swift run -c release argmax-cli tts \
  --text "The old café on the corner still smelled like rain and roasted coffee." \
  --speech-decoder-mode throughputOptimized --play
```

## Performance

Apple M4, single chunk, default compute units. `prod` is the previous (`W8A16`) asset on `main`.

| Text | Variant | Inference | TTFB | SD / step | Speed |
|---|---|---|---|---|---|
| short | prod | 2.68 s | 0.11 s | 21.5 ms | 1.13× |
|  | latencyOptimized | 2.56 s | 0.12 s | 7.8 ms | 1.19× |
|  | **throughputOptimized** | **2.30 s** | 0.28 s | **2.2 ms** | **1.32×** |
| medium | prod | 8.64 s | 0.10 s | 23.8 ms | 1.05× |
|  | latencyOptimized | 7.60 s | 0.10 s | 7.8 ms | 1.19× |
|  | **throughputOptimized** | **6.80 s** | 0.28 s | **2.0 ms** | **1.33×** |
| long | prod | 17.59 s | 0.10 s | 22.5 ms | 1.11× |
|  | latencyOptimized | 16.59 s | 0.12 s | 7.8 ms | 1.18× |
|  | **throughputOptimized** | **14.85 s** | 0.32 s | **2.0 ms** | **1.32×** |

- The new asset cuts per-step SpeechDecoder cost ~3× with no TTFB regression.
- `throughputOptimized` adds another ~4× amortization (~11× total vs. the previous asset), trading ~3× higher TTFB.

## Requirements

The multifunction asset requires **iOS 18 / macOS 15** (gated behind `#available`, the same floor TTSKit already targets).

## Breaking changes

- `SpeechDecoding.decodeFrame{Async}` parameter type changes from `codes: [Int32]` to `codes: [[Int32]]` — external conformers must update.
- The default speech-decoder variant changes from `W8A16` to `W8A16-multifunction`. Callers that explicitly pin `speechDecoderVariant: "W8A16"` should remove the override to pick up the new default; the legacy asset is no longer loadable by this SDK version (older SDK releases pinned to it keep working).

---------

Co-authored-by: Andrey Leonov <aleonov@gmail.com>

* use latest visionos simulator

* Potential fix for pull request finding

---------

Co-authored-by: Eduardo Pacheco <69953243+EduardoPach@users.noreply.github.com>
A
Andrey Leonov committed
8bae6cd4f917eb5109d5d6365106bc8208f7c983
Parent: 0182e2f
Committed by GitHub <noreply@github.com> on 6/25/2026, 10:45:00 PM