Skip to content

Fix snapshots of deleted TsFile resources during DataNode removal - #19192

Open
hongzhi-gao wants to merge 2 commits into
apache:masterfrom
hongzhi-gao:fix/snapshot-deleted-resource
Open

hongzhi-gao wants to merge 2 commits into
apache:masterfrom
hongzhi-gao:fix/snapshot-deleted-resource

Conversation

@hongzhi-gao

@hongzhi-gao hongzhi-gao commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Description

Taking a snapshot captures the TsFile resource lists before closing working files. If an unflushed row has been deleted, closing its empty file marks the resource DELETED and removes the file, but the snapshot's copied list still retains it. Snapshot creation then attempts to capture files that no longer exist. Separately, a missing hard-link source is silently ignored, and deserializing a missing resource file can throw a NullPointerException instead of the declared IOException.

  • Remove DELETED resources from the captured lists and release their snapshot read locks. Continue to capture all live resources; missing files for a live resource fail the snapshot.
  • Propagate a missing hard-link source as FileNotFoundException so an incomplete snapshot cannot be marked complete.
  • Report a null resource input stream as FileNotFoundException, and update the existing English/Chinese log message to remove the claim that missing files are ignored.

No snapshot format or RPC changes.

Validation

Executed against base c6e229aef50608cb67bfb4fd94717d05f4ef3d0f:

  • Targeted DataNode reactor tests: IoTDBSnapshotTest, TsFileResourceTest, and DataRegionTest#testSnapshotAfterDeletingUnflushedData: 20 passed, 1 existing manual multi-disk test skipped.
  • Implementation-time cluster validation: 2 cases passed on 1 ConfigNode + 3 DataNodes with replication factor 1, checking node removal, process exit, and query results for empty-data and retained-data cases. The standalone cluster test harness was removed from the final diff to keep regression coverage focused on the existing component fixtures.
  • Before the fix, the new missing-resource deserialization test fails with NPE, and the missing-resource snapshot rejection test fails because snapshot creation reports success.
  • Spotless, Checkstyle, and focused DataNode reactor test compilation with with-zh-locale passed..

Review

  • Self-reviewed resource-list ownership, read-lock release, and snapshot failure handling.
  • Added unit/component regression coverage.
  • Validated DataNode removal in test clusters.

Key classes: SnapshotTaker, TsFileResource.

@Caideyipi Caideyipi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Reviewed the full diff and the resource-lock and snapshot failure paths at commit 7f64862. Static review only; I did not rerun the tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants