fix(agent): honour an operator's arena limit and check mallopt

Review follow-up on the arena cap.

glibc applies MALLOC_ARENA_MAX and the glibc.malloc.arena_max tunable at
process start, and mallopt() overrides either of them with no way to tell
that one was set. Measured: with MALLOC_ARENA_MAX=1 in the environment a
process settles at 51 MB of virtual size, but an unconditional mallopt(2)
takes it to 109 MB, so the operator's choice is lost rather than honoured.
The cap is now skipped when either is present, which also keeps the knob
available for tuning and for diagnosing allocator behaviour in the field.
An unrelated GLIBC_TUNABLES key still allows the cap.

The return value is now checked instead of discarded. An unchecked
mallopt() is the same pattern Coverity reported across the agent in
#38466, and a future scan would flag it. Failing startup over an
allocator hint would be wrong, so the failure path logs at debug.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>