fix(kafka-python): record actual producer partition from send() future (#4858)
* fix(kafka-python): only record known destination partition in producer spans When neither a key nor an explicit partition is provided, the DefaultPartitioner selects a partition at random. The instrumentation called `_partition()` to record it in the span, but the producer's `send()` calls `_partition()` again independently and may pick a different partition, so `messaging.kafka.partition` frequently did not match where the message was actually delivered. Record the partition only when it can be determined without changing where the message lands (explicit partition, or key-based hash which is deterministic) and omit the attribute for the random keyless case. Fixes #4625 Assisted-by: Claude Opus 4.8 Signed-off-by: allen <allen.k1m@kakaocorp.com> * fix(kafka-python): read actual partition from send() future Instead of estimating the destination partition before send() (which, with the default partitioner and no key, runs a random choice that send() then redoes and diverges from), read the partition back from the FutureRecordMetadata returned by send() via _produce_future.topic_partition. This is accurate for explicit, key-based and random-partition cases alike, and removes the speculative _wait_on_metadata()/_serialize()/_partition() calls the instrumentation previously made. The producer span now wraps the send() call so the partition can be read from its result before the span ends. Fixes #4625 Assisted-by: Claude Opus 4.8 Signed-off-by: allen <allen.k1m@kakaocorp.com> * fix(kafka-python): use PEP 604/585 type hints in partition helpers Address review nits: use `list[str]` and `int | None` for the annotations touched in this change. Assisted-by: Claude Opus 4.8 Signed-off-by: allen <allen.k1m@kakaocorp.com> * Trim extract_send_partition docstring to the essential constraint Move change-justification prose (previous-implementation comparison, issue link) out of the code; that context lives in the PR and #4625. * Remove unused KafkaPropertiesExtractor send key/value helpers * Add kafka-python docker test verifying partition from send() future * Apply ruff format to kafka docker test --------- Signed-off-by: allen <allen.k1m@kakaocorp.com> Signed-off-by: emdneto <9735060+emdneto@users.noreply.github.com> Co-authored-by: emdneto <9735060+emdneto@users.noreply.github.com>
A
Allen Kim committed
38b728363493c62ef1d2be09bf2e71082adc2a97
Parent: 78392b3
Committed by GitHub <noreply@github.com>
on 8/14/2026, 5:35:28 PM