telemetry: add full-pipeline stress tests (#1951)
## What? Test-only, full-pipeline stress coverage for telemetry: concurrent producers through the staging queue → periodic flush → the BUFFER / Prometheus / DOCA exporters. No product, queue, exporter, schema, or public-API changes. - `test/gtest/telemetry_stress_test.cpp` — value parity, `addXferStats` batch atomicity, lifecycle shake-out, safe shutdown (BUFFER, end-to-end). - `test/gtest/telemetry_prometheus_stress_test.cpp` — concurrent staging-overflow conservation and mixed-workload aggregation. - `test/doca-telemetry/telemetry_doca_stress_test.cpp` — high-volume DOCA exporter accumulation (standalone `doca_nixl_stress_test`). ## Why? Closes [NIX-1205](https://linear.app/nvidia/issue/NIX-1205). Existing tests cover queue mechanics and single-threaded exporters, but not the concurrent observable pipeline. This pins down that contract — conservation, all-or-none batching, per-value fidelity, shutdown safety — so a regression anywhere in the path fails an assertion. It builds only on already-merged behavior (NIX-454 #1887, NIX-453 #1919), so it is independent of NIX-1542 and mergeable on its own, while serving as the validation gate for the NIX-1542/NIX-1544 queue changes. ## How? Deterministic and timing-independent: conservation identities and value multisets rather than fixed accepted/dropped splits, bounded polling instead of sleeps, and sanitizer-scaled iteration counts. <details> <summary>Design & validation details</summary> - Under-capacity BUFFER tests size the ring above the total so nothing is dropped; the overflow case drives the lossless, ring-free Prometheus exporter, so `accepted + dropped == produced` is exact and hardware-independent. - `addXferStats` atomicity is proven by per-call identity + contiguity in the drained ring (a torn or interleaved batch fails), not just aggregate counts. - Safe-shutdown destroys on a worker thread bounded by an external timeout, so a teardown deadlock fails as an assertion instead of hanging; producers are released together via a start latch to maximize real concurrency. - Validation: normal C++17, TSan, UBSan, and DOCA-enabled builds pass; focused telemetry suites run warning-clean. ASan can't run in some environments due to a prebuilt mixed-Abseil ABI incompatibility (documented in the plan). </details> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Tests - Added high-volume telemetry stress coverage, including concurrent Prometheus metrics scraping and validation of counters, last-operation gauges, and dropped-event accounting. - Added concurrency tests for overflow handling, metric aggregation, and error/status labeling. - Added end-to-end checks for batching integrity, repeated telemetry lifecycles, and safe shutdown during active flushing. - Added a dedicated DOCA telemetry stress test with an extended timeout. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Efraim Eygin <eeygin@nvidia.com>
E
e-eygin committed
eb26be2e19520768731a8aa4a4c9c94bdf226156
Parent: 039a8b5
Committed by GitHub <noreply@github.com>
on 8/13/2026, 12:11:47 PM