Report a host limit for memories too large for the shell to allocate (#8898)
Fixes the fuzzer finding from #8882 (comment): a maximal memory64 (2^48 pages, exactly 2^64 bytes) hit `shellMemory.resize(memory->initial << memory->pageSizeLog2)` in the shell interface, where the byte size wrapped to 0. Instantiation then "succeeded" with an empty buffer, and every memory access spuriously trapped as out of bounds, including the `memory.init` of active segments during startup. When an optimization legitimately removes such an access (MemoryPacking dropping a provably in-bounds zero segment), the spurious trap disappears and the fuzzer sees a behavior change. The fix reports a host limit at instantiation when the byte size is unrepresentable or the allocation fails, like a real VM whose instantiation fails. This slots into existing machinery: `growMemory` already applies a size limit, and the fuzz-exec harness already catches `HostLimitException` during instance creation and excludes such runs from comparison, precisely because optimizations can change whether a host limit is reached. With this, the reproducer prints `[host limit memory too large]` both before and after `--memory-packing`, so there is no differential. Added as a lit test under `test/lit/exec/`.
J
Jacky Li committed
99f6fc6315d0ae27740211be23b55c9845d504cd
Parent: e2712b6
Committed by GitHub <noreply@github.com>
on 7/13/2026, 11:55:27 PM