Skip to content

fix(cluster): stop partition repair asking for an inverted op range - #4246

Open
hubcio wants to merge 1 commit into
masterfrom
fix/partition-repair-moot-floor
Open

hubcio wants to merge 1 commit into
masterfrom
fix/partition-repair-moot-floor

Conversation

@hubcio

@hubcio hubcio commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

After a leader restart a rejoining backup asked its repair peer for
ops 9..=8, a range header validation rejects. The floor installed by
a RangeEvicted reply is verified on every RepairDone, but the commit
walk can pass it before the reply lands, evicting the headers of the
ops it committed. Such a floor never connects and its window never
looks complete, so the refusal stays InProgress while the walk runs
commit_min to the fetch ceiling, and the reply handler then requests
commit_min + 1 ..= fetch_to_op.

Skip the floor check when the raw floor is at or below commit_min:
it cannot move the commit point and everything below it is applied.
The clamp to commit_max stays inside the check so a peer retaining
nothing this replica needs still escapes to state transfer. The
reply handler also stops pulling a chunk once the walk stands at the
ceiling, the guard the stall retry already had. Making the refusal
definitive instead would arm a state transfer on a replica that
holds every committed op.

Fixes #4230

After a leader restart a rejoining backup asked its repair peer for
ops 9..=8, a range header validation rejects. The floor installed by
a RangeEvicted reply is verified on every RepairDone, but the commit
walk can pass it before the reply lands, evicting the headers of the
ops it committed. Such a floor never connects and its window never
looks complete, so the refusal stays InProgress while the walk runs
commit_min to the fetch ceiling, and the reply handler then requests
commit_min + 1 ..= fetch_to_op.

Skip the floor check when the raw floor is at or below commit_min:
it cannot move the commit point and everything below it is applied.
The clamp to commit_max stays inside the check so a peer retaining
nothing this replica needs still escapes to state transfer. The
reply handler also stops pulling a chunk once the walk stands at the
ceiling, the guard the stall retry already had. Making the refusal
definitive instead would arm a state transfer on a replica that
holds every committed op.

Fixes #4230
@github-actions github-actions Bot added the S-waiting-on-review PR is waiting on a reviewer label Sep 21, 2026
@codecov

codecov Bot commented Sep 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.31%. Comparing base (2ad0b19) to head (373d16b).
⚠️ Report is 3 commits behind head on master.

Additional details and impacted files
@@              Coverage Diff              @@
##             master    #4246       +/-   ##
=============================================
- Coverage     87.50%   69.31%   -18.19%     
  Complexity     1575     1575               
=============================================
  Files          1280     1281        +1     
  Lines        223241   190549    -32692     
  Branches     186604   153913    -32691     
=============================================
- Hits         195337   132088    -63249     
- Misses        23195    53856    +30661     
+ Partials       4709     4605      -104     
Components Coverage Δ
Rust Core 65.92% <95.64%> (-22.65%) ⬇️
Java SDK 68.68% <ø> (ø)
C# SDK 77.41% <ø> (-0.05%) ⬇️
Python SDK 90.97% <ø> (ø)
PHP SDK 85.67% <ø> (ø)
Node SDK 96.43% <ø> (-0.02%) ⬇️
Go SDK 70.18% <ø> (+0.05%) ⬆️
Files with missing lines Coverage Δ
core/partitions/src/iggy_partition.rs 92.81% <100.00%> (-0.01%) ⬇️
core/server/src/partition_reconciler.rs 98.03% <100.00%> (-0.01%) ⬇️
core/shard/src/lib.rs 85.72% <100.00%> (+0.19%) ⬆️

... and 368 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@hubcio

hubcio commented Sep 21, 2026

Copy link
Copy Markdown
Contributor Author

lets wait with merge until author of issue #4230 confirms that PR resolves this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-review PR is waiting on a reviewer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replica recovery panics with “repair range must be non-empty” after requesting 9..=8

4 participants