SIGN IN SIGN UP

Report a host limit for memories too large for the shell to allocate (#8898)

Fixes the fuzzer finding from #8882 (comment): a maximal memory64 (2^48
pages, exactly 2^64 bytes) hit `shellMemory.resize(memory->initial <<
memory->pageSizeLog2)` in the shell interface, where the byte size
wrapped to 0. Instantiation then "succeeded" with an empty buffer, and
every memory access spuriously trapped as out of bounds, including the
`memory.init` of active segments during startup. When an optimization
legitimately removes such an access (MemoryPacking dropping a provably
in-bounds zero segment), the spurious trap disappears and the fuzzer
sees a behavior change.

The fix reports a host limit at instantiation when the byte size is
unrepresentable or the allocation fails, like a real VM whose
instantiation fails. This slots into existing machinery: `growMemory`
already applies a size limit, and the fuzz-exec harness already catches
`HostLimitException` during instance creation and excludes such runs
from comparison, precisely because optimizations can change whether a
host limit is reached.

With this, the reproducer prints `[host limit memory too large]` both
before and after `--memory-packing`, so there is no differential. Added
as a lit test under `test/lit/exec/`.
J
Jacky Li committed
99f6fc6315d0ae27740211be23b55c9845d504cd
Parent: e2712b6
Committed by GitHub <noreply@github.com> on 7/13/2026, 11:55:27 PM