[tiering] Use exponential delay restart strategy for Flink tiering job#3683
Open
Shawn-Hx wants to merge 2 commits into
Open
[tiering] Use exponential delay restart strategy for Flink tiering job#3683Shawn-Hx wants to merge 2 commits into
Shawn-Hx wants to merge 2 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the Flink tiering job entrypoint to use a fixed-delay restart strategy (to avoid Flink’s default no-restart behavior when checkpointing is disabled) and adjusts tiering failover integration testing so it exercises the production entrypoint’s StreamExecutionEnvironment configuration.
Changes:
- Configure
FlussLakeTieringto use Flink’s fixed-delay restart strategy with effectively unlimited restart attempts. - Update
TieringFailoverITCaseto build itsStreamExecutionEnvironmentviaFlussLakeTiering, inject two consecutive lake write failures, and avoid canceling an already-finished job. - Remove checkpoint enabling from the shared tiering IT base and add needed test dependencies to the tiering module.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| fluss-flink/fluss-flink-tiering/src/test/java/org/apache/fluss/flink/tiering/TieringFailoverITCase.java | Reworks failover IT to use the production entrypoint’s exec env and validates recovery from consecutive write failures. |
| fluss-flink/fluss-flink-tiering/src/main/java/org/apache/fluss/flink/tiering/FlussLakeTiering.java | Adds fixed-delay restart strategy configuration for stateless tiering jobs when checkpoints are disabled. |
| fluss-flink/fluss-flink-tiering/pom.xml | Adds test-scoped dependencies needed to run tiering ITs in the tiering module. |
| fluss-flink/fluss-flink-common/src/test/java/org/apache/fluss/flink/tiering/FlinkTieringTestBase.java | Aligns tiering IT base with production by no longer enabling checkpointing by default. |
Comments suppressed due to low confidence (1)
fluss-flink/fluss-flink-tiering/src/test/java/org/apache/fluss/flink/tiering/TieringFailoverITCase.java:75
- This test bootstraps the ExecutionEnvironment via FlussLakeTiering, but passes
--datalake.formatas Paimon while the actual tiering job built in FlinkTieringTestBase is hard-coded to useDataLakeFormat.LANCE.toString()(see FlinkTieringTestBase#buildTieringJob). Using LANCE here keeps the test configuration consistent and avoids masking format-specific behavior in FlussLakeTiering initialization.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
luoyuxia
reviewed
Jul 18, 2026
luoyuxia
reviewed
Jul 18, 2026
Configure the stateless tiering job to retry indefinitely when checkpointing is disabled, preventing transient failures from terminating the job permanently. Move the failover integration test to fluss-flink-tiering so that it covers the production job configuration.
Explain why flink-table-common must be present on the fluss-flink-tiering test runtime classpath.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Configure the stateless tiering job to retry indefinitely when checkpointing is disabled, preventing transient failures from terminating the job permanently.
Move the failover integration test to fluss-flink-tiering so that it covers the production job configuration.
Purpose
Linked issue: close #3681
Brief change log
Tests
TieringFailoverITCaseAPI and Format
No public API or storage format changes.
Documentation
No documentation changes are required.