Merge master into feat/18479-kafka-streams-runner-skeleton - #39791
Closed
junaiddshaukat wants to merge 744 commits into
Closed
Merge master into feat/18479-kafka-streams-runner-skeleton#39791junaiddshaukat wants to merge 744 commits into
junaiddshaukat wants to merge 744 commits into
Conversation
…apache#39385) Bumps [github.com/fsouza/fake-gcs-server](https://github.com/fsouza/fake-gcs-server) from 1.52.3 to 1.55.1. - [Release notes](https://github.com/fsouza/fake-gcs-server/releases) - [Commits](fsouza/fake-gcs-server@v1.52.3...v1.55.1) --- updated-dependencies: - dependency-name: github.com/fsouza/fake-gcs-server dependency-version: 1.55.1 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Fix pr-bot-new-prs
…pache#39351) * Add --add-opens test JVM args for HCatalog and Dataflow worker tests * add more --add-opens for Dataflow * document --add-opens for WindmillStateTestUtils cache * '--add-opens=java.base/java.util.concurrent.locks=ALL-UNNAMED' * Add remaining --add-opens for WindmillStateTestUtils * fix worker tests for JDK 21 Thread.toString format
* Support ACKNOWLEDGEMENT_MODE for JmsIO * CheckpointMark behavior in alignment with different ACKNOWLEDGEMENT_MODE * Ref count active checkpoint for quicker onClose that releases session * Fix hanging checkpoint when no incoming data in direct runner. This allows us to do an exact assert * Optimize long running unit test usign a short retry * Re-enable AMQP integration test after stuck unack messages resolved * Address comments * Loose withCloseTimeout to fix flaky integration test; fix label * Fix javadoc
* Cache DoFn invoker constructors per instance Co-Authored-By: Codex <noreply@openai.com> * Add DoFn invoker cache fix to changelog Co-Authored-By: Codex <noreply@openai.com> * Clarify DoFn cache performance improvement Co-Authored-By: Codex <noreply@openai.com> --------- Co-authored-by: Codex <noreply@openai.com>
* add language filtering for codeql * fix workflow file * remove third party dorny action * add comments * use approved SHA of dorny/paths-filter to dynamically filter matrix * add checkout step before paths-filter * compact ALL_MATRIX JSON to a single line before writing to GITHUB_OUTPUT * update path filters to use file extensions and build configuration files * remove uncessary checks
…#39284) * Remove support for Kafka clients older than 3.9.2 * Resolve capability conflicts for Flink 1.x, Flink 2.0, Spark 3 and Spark 4 * Resolve capability conflicts for load tests and watermarks * Replace obsolete signature of overridden method close in mock consumer * Replace obsolete class KafkaServerStartable with KafkaServer * Fix type ambiguity of method argument in test * Handle Kafka and executor timeouts the same
Bumps [protobufjs](https://github.com/protobufjs/protobuf.js) from 8.6.0 to 8.6.6. - [Release notes](https://github.com/protobufjs/protobuf.js/releases) - [Changelog](https://github.com/protobufjs/protobuf.js/blob/master/CHANGELOG.md) - [Commits](protobufjs/protobuf.js@protobufjs-v8.6.0...protobufjs-v8.6.6) --- updated-dependencies: - dependency-name: protobufjs dependency-version: 8.6.6 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [pillow](https://github.com/python-pillow/Pillow) from 10.2.0 to 12.3.0. - [Release notes](https://github.com/python-pillow/Pillow/releases) - [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst) - [Commits](python-pillow/Pillow@10.2.0...12.3.0) --- updated-dependencies: - dependency-name: pillow dependency-version: 12.3.0 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 7. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v4...v7) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [pillow](https://github.com/python-pillow/Pillow) from 10.2.0 to 12.3.0. - [Release notes](https://github.com/python-pillow/Pillow/releases) - [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst) - [Commits](python-pillow/Pillow@10.2.0...12.3.0) --- updated-dependencies: - dependency-name: pillow dependency-version: 12.3.0 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Abdelrahman Ibrahim <aeldsouky.c>
* Sanitize the exception message stored for failed instructions to avoid inadvertently capturing the content of stack frames in the heap and limit RAM growth due to cache size. * Use Py3.10 syntax
* Update CHANGES.md * Fix AI review comments
… fallback in examples
IcerbergIO Load test and benchmarking
…#39401) Co-authored-by: Abdelrahman Ibrahim <aeldsouky.c>
…e#39390) * Fix gRPC stream observer leak on ProcessBundleHandler shutdown * tears down active gRPC multiplexers on ProcessBundleHandler shutdown * Update sdks/java/harness/src/main/java/org/apache/beam/fn/harness/data/BeamFnDataGrpcClient.java Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* add third party allow list check and doc * add python action and shift validate further down * fix two workflow/action issues caught during test run * reuse setup environment step per comment
* [Gemini] Add Java GeminiModelHandler class * Remove model scratch files * address review comments * checkstyle issue * unit test fix * Fix weird coder UX * fix altered base class signatures
* Add metric for Kafka offset commit failures * Apply suggestions from code review Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: Jack McCluskey <34928439+jrmccluskey@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Add drain states to PipelineResult * Fix draining job lookup for Dataflow updates
Bumps [github.com/aws/aws-sdk-go-v2/credentials](https://github.com/aws/aws-sdk-go-v2) from 1.19.29 to 1.19.30. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Commits](aws/aws-sdk-go-v2@credentials/v1.19.29...credentials/v1.19.30) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/credentials dependency-version: 1.19.30 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Create beam-summit-2026-interview-with-raj-katakam.md Interview with Raj Katakam * Update authors.yml Added my info to authors.yml * Update beam-summit-2026-interview-with-raj-katakam.md * Update website/www/site/content/en/blog/beam-summit-2026-interview-with-raj-katakam.md --------- Co-authored-by: Danny McCormick <dannymccormick@google.com>
Add documentation comments to previously undocumented messages and fields in the Beam job management and interactive API protos: beam_artifact_api.proto: ArtifactRetrievalService, ArtifactStagingService, ArtifactRequestWrapper, ProxyManifest, PutArtifactMetadata, and more. beam_expansion_api.proto: ExpansionRequest/Response, DiscoverSchemaTransformRequest/Response, SchemaTransformConfig. beam_job_api.proto: JobService, JobInfo, JobStateEvent, JobMessage, GetJobMetricsRequest/Response, MetricResults, PipelineOptionType. beam_interactive_api.proto: TestStreamService, EventsRequest.
…pache#39402) * Speed up Python container builds with prefer-binary and registry cache * Address review * Drop docker registry cache --------- Co-authored-by: Abdelrahman Ibrahim <aeldsouky.c>
…#39421) Bumps [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) from 1.22.34 to 1.22.35. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Commits](aws/aws-sdk-go-v2@feature/s3/manager/v1.22.34...feature/s3/manager/v1.22.35) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager dependency-version: 1.22.35 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Derrick Williams <derrickaw@google.com>
Contributor
|
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment |
Contributor
Author
|
Superseded — when merged master into the branch directly, which keeps the history cleaner than a merge of a merge. Thanks. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings the feature branch up to date with master, as discussed, so the diff on #39785 shows only what the runner adds rather than master's own changes appearing as removals.
The merge is clean, nothing conflicted.
Verified after it:
./gradlew -Pwith-kafka-streams-runner :runners:kafka-streams:buildpasses against current mastervalidatesRunnerpasses, 59 tests./gradlew projectsstill lists no Kafka Streams projects without the flag, and all four with ittox.inikeeps master'spyarrow-{17,...,25}environments alongside ourkafka-streams-runner-test, which was the thing that looked alarming in the two-dot diff