feat(promhttp): add CoalesceGather option to deduplicate concurrent Gather calls (#1969)

* feat(promhttp): add CoalesceGather option to deduplicate concurrent Gather calls

When a collector's Collect() is slower than the scrape interval, each
incoming HTTP request triggers an independent Gather() that spawns its
own goroutine pipeline. With no upper bound, this causes goroutine
pile-up proportional to (scrape rate / collection time) — the apparent
"goroutine leak" reported in #1477.

Add HandlerOpts.CoalesceGather bool. When true, HandlerForTransactional
wraps the underlying TransactionalGatherer in a coalescingGatherer that
allows only one Gather to run at a time. Concurrent requests join the
in-flight cycle and receive the same result once it completes. Reference
counting ensures the TransactionalGatherer done() callback is called
exactly once, after the last handler finishes encoding.

Design decisions:
- Per-cycle gatherCycle object (not fields on the struct) prevents the
  race where a new cycle overwrites result fields while prior waiters are
  still reading them.
- Mutex-based ref counting (not atomic) ensures c.cycle = nil is cleared
  before cy.done() is called, ruling out double-done on a stale pointer.
- close(cy.ready) happens-before <-cy.ready returns (Go memory model),
  so cy.mfs/err/done are safely readable without additional locking.
- Zero overhead when disabled: single if-branch in HandlerForTransactional
  setup, identical code path when CoalesceGather is false.

Fixes #1477

Signed-off-by: Kemal Akkoyun <kemal.akkoyun@datadoghq.com>

* fix(promhttp): fail coalesced joiners when the gatherer panics

When CoalesceGather is enabled and the wrapped gatherer itself panics, requests that joined the in-flight cycle were unblocked with a nil error and returned an empty but successful response. They now receive a sentinel error and fail like the panicking request instead of silently reporting no metrics. The leader's panic still propagates and is handled by net/http, matching the non-coalesced path; no recover is introduced.

Also document the panic behaviour on the option, make the coalescing invariant test deterministic instead of timing-dependent, and align the goroutine-leak test with the repository's WaitGroup convention.

Signed-off-by: Kemal Akkoyun <kemal.akkoyun@datadoghq.com>

* fix(promhttp): give coalesced callers independent metric slices

Concurrent callers sharing a coalesced Gather cycle previously received the same slice header. A caller that filtered or reordered its slice (for example applying name[] query parameters) could then race another caller of the same cycle. Each caller now gets its own slices.Clone; the MetricFamily values stay shared and read-only.

Also clarify the CoalesceGather documentation (shared-snapshot staleness, the Timeout interaction, and slice-versus-contents mutation) and tidy the sentinel error and coalescer test helpers.

Signed-off-by: Kemal Akkoyun <kemal.akkoyun@datadoghq.com>

* docs(promhttp): mark CoalesceGather as experimental

Signed-off-by: Kemal Akkoyun <kemal.akkoyun@datadoghq.com>

---------

Signed-off-by: Kemal Akkoyun <kemal.akkoyun@datadoghq.com>