SIGN IN SIGN UP
oven-sh / bun UNCLAIMED

Incredibly fast JavaScript runtime, bundler, test runner, and package manager – all in one

0 0 150 Rust

js_parser: restore SIMD block comment skipping via Highway (#31433)

### What does this PR do?

Restores the block-comment SIMD skip that the Rust port of the JS lexer
lost, by moving it to the same Google Highway kernels the lexer already
uses for single-line comments.

**Before this PR**

- `scan_single_line_comment` kept its SIMD path in the port
(`bun_highway::index_of_newline_or_non_ascii_or_hash_or_at`).
- `scan_multi_line_comment_body` kept the Zig structure (only take the
fast path while the current code point is ASCII and ≥ 512 bytes remain),
but the inner scan — Zig's
`skipToInterestingCharacterInMultilineComment`, a 16-byte `@Vector` loop
— was ported as a scalar byte-at-a-time loop with a `TODO(port): SIMD
reimplementation` note (`src/js_parser/lexer.rs`). Large license headers
/ JSDoc blocks were scanned one byte per iteration.

**This PR**

- Adds an `IndexOfInterestingCharacterInMultilineCommentImpl` kernel to
`src/jsc/bindings/highway_strings.cpp` (same `HWY_EXPORT` +
`HWY_DYNAMIC_DISPATCH` pattern as the existing kernels). It returns the
index of the first `*` (potential `*/` terminator), `\r`, `\n` (newline
tracking for ASI), or non-ASCII byte (so U+2028/U+2029 and other
multi-byte sequences are still decoded by the scalar path) — exactly the
byte classes the Zig `@Vector` version stopped at.
- Exposes it as
`bun_highway::index_of_interesting_character_in_multiline_comment` (with
the same debug-only result validation as the neighboring wrappers).
- `skip_to_interesting_character_in_multiline_comment` in the lexer now
calls it instead of the scalar loop. The `Environment::ENABLE_SIMD` gate
at the call site is dropped: it exists for portable-vector codegen, but
Highway dispatches per-CPU at runtime, so baseline builds take the fast
path too — matching how `scan_single_line_comment` and the
string-literal scan already call Highway unconditionally. The ≥ 512-byte
threshold and the ASCII-code-point check are unchanged.
- Adds the new per-target kernel symbols to the `verify-baseline-static`
allowlists (x64, x64-windows, aarch64), with feature ceilings copied
from the structurally identical `IndexOfNewlineOrNonASCIIImpl` entries.
If the baseline CI scan reports different feature sets, I'll update the
entries to match its report.

### Benchmarks

`Bun.Transpiler.transformSync` on linux x64 (AVX2), comparing release
builds of main (`19dd34df3`) and this branch (`84fe8aa26`) built
back-to-back from this tree. Interleaved runs, median of 9 inner rounds,
two outer rounds per binary (values were stable to ~1%); transpiled
output hashes are identical between the two binaries for every case.
Shared container, so ratios are the signal.

| case | main | this PR | Δ |
| --- | --- | --- | --- |
| 200 × ~2 KB JSDoc block comments + 200 small fns (385 KB) | 1.071 ms |
0.834 ms | **1.28× faster** |
| 1 MB block comment, ~100-char lines | 1.331 ms | 0.822 ms | **1.62×
faster** |
| 1 MB block comment, ~1 KB lines | 1.300 ms | 0.681 ms | **1.91×
faster** |
| 16 KB license header + 200 fns (31 KB total) | 0.476 ms | 0.464 ms |
~2% (run time dominated by the code, not the comment) |
| control: same 200 fns, no comments | 0.443 ms | 0.438 ms | parity |
| control: 200 fns each behind a `//` line comment | 0.439 ms | 0.439 ms
| parity |

The two controls confirm the delta is attributable to the block-comment
path (nothing else in the lexer changed between the two revisions).

### How did you verify your code works?

All of the following ran against the debug (ASAN) build with this
change:

- `bun bd test test/bundler/transpiler/transpiler.test.js` — 165 pass /
0 fail, including a new `multi-line comment scanning` group:
comment-size sweep across the 512-byte threshold and vector-width
boundaries, a lone `*` at every offset in the first 80 bytes of a large
comment, all-`*` bodies, `\n` / `\r` / `\r\n` / U+2028 / U+2029 inside
large comments observable through ASI, non-ASCII bodies (2-byte, 4-byte,
mixed), comments ending exactly at EOF, and unterminated large comments
producing `Expected "*/" to terminate multi-line comment`.
- `bun bd test test/bundler/bundler_comments.test.ts` — 45 pass / 0
fail, including new end-to-end bundles: a large `/*! ... */` legal
comment (CRLF + non-ASCII) preserved verbatim with the code after it
intact, and ASI behavior across a large block comment verified by
running the bundled output.
- `bun bd test test/js/bun/transpiler/transpiler-truncated-utf8.test.ts`
— the guard-page fixture now also places ≥ 512-byte block comments
(terminated, unterminated, trailing `*`, trailing truncated UTF-8 lead)
so that the input ends exactly at a `PROT_NONE` page; any read past the
end of the source by the new kernel faults deterministically.
- `cargo clippy -p bun_highway -p bun_js_parser` clean, `cargo fmt`
applied, `highway_strings.cpp` is clang-format clean.
- The expected outputs for every new test were cross-checked against the
current scalar implementation before making the change, so this is
verified behavior-preserving. Like #31385, the new tests are regression
coverage for the rewritten path rather than a
failing-before/passing-after proof — there is no functional delta to
assert for a pure performance restoration; the before/after evidence for
the optimization itself is the benchmark table above.
R
robobun committed
f958cbd552d7bcc3d0e7de1d9d077d3f96e07ba0
Parent: c78d08f
Committed by GitHub <noreply@github.com> on 5/26/2026, 9:24:50 PM