SIGN IN SIGN UP
oven-sh / bun UNCLAIMED

Incredibly fast JavaScript runtime, bundler, test runner, and package manager – all in one

0 0 150 Rust

TextDecoder: keep the codec alive across streaming chunks for legacy multi-byte encodings (#31438)

### What does this PR do?

Streaming `TextDecoder` (`{stream: true}`) of every multi-byte
legacy/CJK encoding — shift_jis, euc-jp, iso-2022-jp, gbk, gb18030,
big5, euc-kr — silently corrupted any character or escape state split
across a chunk boundary (e.g. big5 `[A4][40]` → `"@"` instead of `"一"`).
Whole (non-streaming) decode was correct; utf-8/utf-16/latin1 were
unaffected.

The catch-all decode arm created a fresh WebKit `TextCodec` on every
`decode()` call and destroyed it at scope exit. The codec is where the
streaming state lives (lead byte, ISO-2022-JP mode, GB18030
first/second/third), so that state was thrown away between chunks and
the next chunk's trail bytes decoded standalone.

Persist the codec on the `TextDecoder` across `{stream: true}` chunks
(lazy create, reuse, drop+recreate after a flushing decode per the
WHATWG spec, free in `Drop`). This mirrors WebKit's own `TextDecoder`,
which keeps a single `m_codec` and calls `decode(..., flush)`
repeatedly.

Also makes the (currently ungenerated) DOMJIT fast path
`decode_without_type_checks` flush like the slow path does for a
no-options `decode(buf)`, so it cannot carry persisted codec state
across independent decodes if DOMJIT signatures are re-enabled.

Not a port regression — the Zig path has the same shape — but Node and
the spec are correct.

### How did you verify your code works?

- New matrix of encoding × split-offset streaming tests plus a
reuse-after-flush test in
`test/js/web/encoding/text-decoder-cjk.test.ts`; every expected value
cross-checked against Node.js. They fail on current bun (16 failures)
and pass with this branch (30/30).
- The rest of `test/js/web/encoding/` passes with the branch.

---------

Co-authored-by: robobun <robobun@oven.sh>
A
Alistair Smith committed
bb14c4e0215c86bc11f7cc59cb38c4c335117276
Parent: 13bbb38
Committed by GitHub <noreply@github.com> on 5/27/2026, 6:04:04 PM