Fix strict evaluation for negative record counts - #3831
Merged
kevinjqliu merged 1 commit intoAug 23, 2026
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes strict metrics evaluation for negative record counts, preventing unsafe whole-file deletion decisions.
Changes:
- Treat zero-record files as empty.
- Treat negative counts conservatively as potentially matching rows.
- Update regression coverage.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
tests/expressions/test_evaluator.py |
Verifies conservative handling of negative counts. |
pyiceberg/expressions/visitors.py |
Corrects strict record-count handling. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
geruh
approved these changes
Aug 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
record_count == 0as empty in strict metrics evaluation.ROWS_MIGHT_NOT_MATCH.This prevents strict evaluation from classifying files with unknown counts as fully matching, which can incorrectly authorize whole-file deletion.
Java validates newly built files with
recordCount >= 0(DataFiles.java), while manifest reads instantiate through Avro (GenericDataFile.java). Its inclusive evaluator handles negative counts as unknown (InclusiveMetricsEvaluator.java); the strict evaluator still groups them with empty files (StrictMetricsEvaluator.java).Related to #3498.
Tests
uv run pytest tests/expressions/test_evaluator.py -q