[WIP][GSoC 2026] Merge the Kafka Streams runner into master - #39770

Closed

junaiddshaukat wants to merge 39 commits into

apache:masterfrom

junaiddshaukat:feat/ks-runner-to-master

Commits on May 19, 2026

Add Kafka Streams runner skeleton module and portable entry points
junaiddshaukat
committed
2e44530
Address review notes on KafkaStreamsPipelineResult and state dir
junaiddshaukat
committed
61c891a

Commits on May 21, 2026

Address review feedback on Kafka Streams Runner skeleton
junaiddshaukat
committed
cef6544

Commits on May 22, 2026

Drop setRunner(null) suppression; make applicationId required
junaiddshaukat
committed
0d445f7

Commits on May 23, 2026

Commits on May 28, 2026

Commits on Jun 7, 2026

[GSoC 2026] Kafka Streams runner — ExecutableStage (stateless ParDo) translator (#38764)
junaiddshaukat
authored
ff554f7

Commits on Jun 10, 2026

[GSoC 2026] Kafka Streams runner — Redistribute translator + ExecutableStage type-agnostic edge (#38843)
junaiddshaukat
authored
4c02a77

Commits on Jun 15, 2026

#38957: Add in-memory WatermarkManager core (per-source-partition tracking
junaiddshaukat
authored
4636b1f

Commits on Jun 18, 2026

[GSoC 2026] Kafka Streams runner #38987: Wire WatermarkManager into ExecutableStageProcessor
junaiddshaukat
authored
1058e94

Commits on Jun 27, 2026

[GSoC 2026] Kafka Streams runner #39051: Add KStreamsPayload Serde for crossing topic boundaries
junaiddshaukat
authored
3e963a6

Commits on Jun 30, 2026

[GSoC 2026] Kafka Streams runner #39141: Add GroupByKey (GlobalWindow, fire at watermark)
junaiddshaukat
authored
5e65d47

Commits on Jul 7, 2026

[GSoC 2026] Kafka Streams runner #39211: Add KafkaStreamsTestRunner test harness
junaiddshaukat
authored
4d5847f

Commits on Jul 9, 2026

[GSoC 2026] Kafka Streams runner #39249: Support Create
junaiddshaukat
authored
27c4522

Commits on Jul 15, 2026

[GSoC 2026] Kafka Streams runner #39273: Kafka Streams runner: Flatten support
junaiddshaukat
authored
ff323eb

Commits on Jul 16, 2026

[GSoC 2026] Kafka Streams runner: surface SDK-harness metrics as MetricResults (#39341)
junaiddshaukat
authored
75adf49

Commits on Jul 17, 2026

[GSoC 2026] Kafka Streams runner: TestPipeline-dispatchable test runner (PAssert works) (#39362)
junaiddshaukat
authored
87911f0

Commits on Jul 20, 2026

[GSoC 2026] Kafka Streams runner: validatesRunner task; Create and Flatten suites green (#39380)
junaiddshaukat
authored
d9cea58

Commits on Jul 22, 2026

[GSoC 2026] Kafka Streams runner: multi-output executable stages (#39410)
junaiddshaukat
authored
fca1435

Commits on Jul 23, 2026

[GSoC 2026] Kafka Streams runner: enable ParDoTest in the ValidatesRunner suite (#39451)
junaiddshaukat
authored
b615ae8

Commits on Jul 28, 2026

[GSoC 2026] Kafka Streams runner: windowed GroupByKey via ReduceFnRunner (#39494)
junaiddshaukat
authored
47614a9

Commits on Aug 1, 2026

[GSoC 2026] Kafka Streams runner: run on a real broker, correctly across partitions (#39546)
junaiddshaukat
authored
2719876

Commits on Aug 3, 2026

[GSoC 2026] Kafka Streams runner: bound a bundle by element count (#39578)
junaiddshaukat
authored
cf1f10b

Commits on Aug 4, 2026

Commits on Aug 9, 2026

[GSoC 2026] Kafka Streams runner: user documentation, marked experimental (#39627)
junaiddshaukat
authored
cb30afd

Commits on Aug 10, 2026

[GSoC 2026] Kafka Streams runner: Python wrapper that starts its own job server (#39680)
junaiddshaukat
authored
e051e06

Commits on Aug 11, 2026

[GSoC 2026] Kafka Streams runner: terminate a bounded pipeline when it is drained (#39700)
junaiddshaukat
authored
4ff6180

Commits on Aug 13, 2026

[GSoC 2026] Kafka Streams runner: portable ValidatesRunner suite for Python (#39736)
junaiddshaukat
authored
5f1658f

Commits on Aug 14, 2026

[GSoC 2026] Kafka Streams runner: separate the source's poll size from the bundle size, and expose the session timeout (#39748)
junaiddshaukat
authored
511a40e

Commits on Aug 16, 2026

Commits on Aug 17, 2026

[GSoC 2026] Kafka Streams runner: shorten the explanation comments
junaiddshaukat
committed
c5d8006