[labrenbe]

Part of stackabletech/issues#888.

Stacked on #1028.

Description

• New crate stackable-kafka-admin with the requests for describing, creating and deleting topics, adding partitions, reading and changing topic configs, and listing partition reassignments.
• Plaintext, TLS and mutual TLS connections to the bootstrap servers, with the certificates read from files on every new connection, so rotated certificates are picked up without a restart.
• One connection at a time: it is reused across requests, replaced after idling, reconnected after a failed request, and a request that hits a connection the broker already closed is sent once more.
• API versions are pinned to what Kafka 3.9.2 and 4.3.1 both support and checked against the versions the broker advertises.
• Errors distinguish a request the broker rejected, with the Kafka error code, from a connection problem, so the caller can report them differently.
• A shared crate carries the contract between operator and agent: how the operator configures the agent's connection to Kafka through environment variables.
• Unit tests run the client against a mock broker on a local socket.

Out-of-scope

• Everything the agent does not need: ACLs, consumer groups, quotas, broker configs, producing and consuming, explicit replica placement, and partition reassignments beyond listing the running ones.
• ZooKeeper clusters. Requests go to a bootstrap broker, which in KRaft mode forwards them to the controller.
• SASL authentication, including Kerberos. Only anonymous, TLS and mutual TLS are supported.
• API version negotiation. The versions are pinned to what Kafka 3.9.2 and 4.3.1 both support and checked against the broker.
• Connection management beyond one connection with one request at a time and a single retry on a closed connection: no pooling, no backoff, no metrics.

Definition of Done Checklist

• Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
• Please make sure all these things are done and tick the boxes

Author

• Changes are OpenShift compatible
• CRD changes approved
• CRD documentation for all fields, following the style guide.
• Helm chart can be installed and deployed operator works
• Integration tests passed (for non trivial changes)
• Changes need to be "offline" compatible
• Links to generated (nightly) docs added
• Release note snippet added

Reviewer

• Code contains useful comments
• Code contains useful logging statements
• (Integration-)Test cases added
• Documentation added or updated. Follows the style guide.
• Changelog updated
• Cargo.toml only contains references to git tags (not specific commits or branches)

Acceptance

• Feature Tracker has been updated
• Proper release label has been added
• Links to generated (nightly) docs added
• Release note snippet added
• Add type/deprecation label & add to the deprecation schedule
• Add type/experimental label & add to the experimental features tracker

[maltesander]

LGTM!

The base branch was changed.

[maltesander]

LGTM!