bundle: Preallocate buffers for file contents. (#6818)
This commit adds logic to preallocate buffers when loading files from both tarballs and on-disk bundle directories. The change results in lower max RSS memory usage at runtime, and better garbage collector performance, especially when at lower values of GOMAXPROCS. For very large bundles (>1 GB in size), this change can lower startup times for OPA by as much as a full second. The performance analysis was different than for most changes-- heap usage increased by about 10% during bundle loading, which made the change look bad at first. Some of the effect appears to be from the Go compiler no longer inlining as far up the call chain during bundle loading (visible in the `pprof` graphs). Running with `GODEBUG=gctrace=1` and varying GOMAXPROCS allowed seeing a fuller picture of how performance changes from preallocation, which results in much less garbage for the collector, and a noticeable speedup in wall-clock time the GC burns during bundle loading. Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
P
Philip Conrad committed
8e7172c8f8e0b03f654d66aeb95762d8220e46e8
Parent: 3b06458
Committed by GitHub <noreply@github.com>
on 6/19/2024, 12:19:15 PM