SIGN IN SIGN UP

fix(kafka-python): record actual producer partition from send() future (#4858)

* fix(kafka-python): only record known destination partition in producer spans

When neither a key nor an explicit partition is provided, the
DefaultPartitioner selects a partition at random. The instrumentation
called `_partition()` to record it in the span, but the producer's
`send()` calls `_partition()` again independently and may pick a
different partition, so `messaging.kafka.partition` frequently did not
match where the message was actually delivered.

Record the partition only when it can be determined without changing
where the message lands (explicit partition, or key-based hash which is
deterministic) and omit the attribute for the random keyless case.

Fixes #4625

Assisted-by: Claude Opus 4.8
Signed-off-by: allen <allen.k1m@kakaocorp.com>

* fix(kafka-python): read actual partition from send() future

Instead of estimating the destination partition before send() (which,
with the default partitioner and no key, runs a random choice that send()
then redoes and diverges from), read the partition back from the
FutureRecordMetadata returned by send() via _produce_future.topic_partition.
This is accurate for explicit, key-based and random-partition cases alike,
and removes the speculative _wait_on_metadata()/_serialize()/_partition()
calls the instrumentation previously made.

The producer span now wraps the send() call so the partition can be read
from its result before the span ends.

Fixes #4625

Assisted-by: Claude Opus 4.8
Signed-off-by: allen <allen.k1m@kakaocorp.com>

* fix(kafka-python): use PEP 604/585 type hints in partition helpers

Address review nits: use `list[str]` and `int | None` for the annotations
touched in this change.

Assisted-by: Claude Opus 4.8
Signed-off-by: allen <allen.k1m@kakaocorp.com>

* Trim extract_send_partition docstring to the essential constraint

Move change-justification prose (previous-implementation comparison,
issue link) out of the code; that context lives in the PR and #4625.

* Remove unused KafkaPropertiesExtractor send key/value helpers

* Add kafka-python docker test verifying partition from send() future

* Apply ruff format to kafka docker test

---------

Signed-off-by: allen <allen.k1m@kakaocorp.com>
Signed-off-by: emdneto <9735060+emdneto@users.noreply.github.com>
Co-authored-by: emdneto <9735060+emdneto@users.noreply.github.com>
A
Allen Kim committed
38b728363493c62ef1d2be09bf2e71082adc2a97
Parent: 78392b3
Committed by GitHub <noreply@github.com> on 8/14/2026, 5:35:28 PM