Skip to content

Add read-only config attribute to Consumer, Producer, ShareConsumer - #2342

Open
Paradox (i-am-paradox) wants to merge 2 commits into
confluentinc:masterfrom
i-am-paradox:read-client-config
Open

Add read-only config attribute to Consumer, Producer, ShareConsumer#2342
Paradox (i-am-paradox) wants to merge 2 commits into
confluentinc:masterfrom
i-am-paradox:read-client-config

Conversation

@i-am-paradox

Copy link
Copy Markdown

This adds a read-only config attribute to Consumer, Producer and ShareConsumer, as requested in #465. The AIO wrappers keep their own composition style and are not touched.

The attribute returns a dict of the effective configuration of the instance: it is built from rd_kafka_conf() on the client handle followed by rd_kafka_conf_dump(), so it shows the values actually in use, including librdkafka defaults applied on top of the dict passed to the constructor. The dict is a fresh copy on every access, mutating it does not affect the client.

Two notes on the behaviour:

  • Properties that carry internal non-string values in librdkafka's dump are excluded, since they are implementation details of the bindings: the various *_cb pointers set by the bindings, opaque, socket_cb and open_cb. Callbacks passed by the application in the config dict never appear in the dump anyway.
  • Values are strings, that is what rd_kafka_conf_dump() returns. client.id comes back as the configured string, session.timeout.ms as '1000' and so on. I kept them as-is rather than trying to parse known numeric keys, the round-trip through int() looked like more guessing than value, but I can convert the well-known numeric properties if that's preferred.

bootstrap.servers can come back as None in the dump when it was consumed into the instance, that is how librdkafka reports it, I left the value untouched.

Tests are in tests/test_misc.py and run without a broker: reading the config of a Producer and a Consumer and checking configured values against the dump, checking that a librdkafka default (api.version.request) is present, that the dict is a detached copy, that the attribute has no setter, and that error_cb passed in the config dict does not show up.

On this machine the unit test files give the same result before and after this change (36 pre-existing failures in the AIO tests, they need dependencies I don't have set up locally), with the 4 new tests passing on top.

Fixes #465

Expose the effective configuration of a client instance through a
read-only config attribute, so applications can inspect the values
actually in use without tracking the dict passed to the constructor.

The dict is built from rd_kafka_conf() on the client instance and
rd_kafka_conf_dump(), so it reflects librdkafka defaults applied on
top of the user configuration. Properties that hold internal
non-string values in the dump (callback pointers, opaque) are left
out, they are bindings implementation details.

Values are reported as strings, which is what librdkafka's config
dump returns.

Fixes confluentinc#465
@confluent-cla-assistant

Copy link
Copy Markdown

Please sign the Contributor License Agreement here before this PR can be approved.
❌ i-am-paradox
Please push an empty commit if you would like to re-run the checks to verify CLA status for all contributors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

read access to consumer/producer configuration

1 participant