Skip to content

Prevent tree last-cache updates from restoring deleted schemas - #19171

Open
Caideyipi wants to merge 1 commit into
masterfrom
codex/pr-tree-last-cache-schema-isolation
Open

Caideyipi wants to merge 1 commit into
masterfrom
codex/pr-tree-last-cache-schema-isolation

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Summary

Prevent tree-model LAST cache initialization, update, and invalidation from repopulating the schema cache after schema deletion. Schema cache population remains on the protected schema acquisition and creation paths.

Available incident logs do not establish that a concurrent LAST query triggered the reported symptom.

Known limitation and follow-up

An older LAST query can still publish its result to the last-value cache after a same-name schema is dropped and recreated. Complete protection requires capturing a schema-generation token during planning, propagating it through distributed plan serialization, and validating it during cache initialization and asynchronous completion on every DataNode and Region.

That path crosses schema metadata lifecycle, distributed plan transport, concurrent cache state, and multi-node/multi-Region tests. It is substantially broader than separating schema-cache writes from last-cache updates, so it is deferred to a dedicated follow-up.

Changed paths

  • iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/metadata/fetcher/cache/TableDeviceCacheEntry.java
  • iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/metadata/fetcher/cache/TableDeviceSchemaCache.java
  • iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/metadata/fetcher/cache/TreeDeviceSchemaCacheManager.java
  • iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/last/AbstractUpdateLastCacheOperator.java
  • iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/plan/node/write/InsertRowNode.java
  • iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/plan/node/write/InsertTabletNode.java
  • iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/DataRegion.java
  • iotdb-core/datanode/src/test/java/org/apache/iotdb/db/metadata/cache/TreeDeviceSchemaCacheManagerTest.java

Verification

  • mvn spotless:apply -P with-integration-tests -pl iotdb-core/datanode passed.
  • git diff --check passed.
  • Local clean DataNode reactor tests passed: 20 tests, 0 failures, 0 errors; DataNode main and test sources compiled.
  • Full server integration tests, master runtime validation, and field validation were not run.

@Caideyipi

Copy link
Copy Markdown
Collaborator Author

Initial self-review summary

  • Finding: An older LAST query may still update the last-value cache after a same-name schema is dropped and recreated. Available logs do not establish that this race caused the reported symptom. A complete generation guard crosses planning, distributed plan transport, cache concurrency, and multi-node/multi-Region tests, so it is deferred as documented in the PR.
  • Fix: Tree last-cache initialization, update, and invalidation no longer repopulate schema cache; protected schema acquisition and creation paths remain responsible for schema population.
  • Verification: Spotless and git diff --check passed. Local clean DataNode reactor tests passed: 20 tests, 0 failures, 0 errors, with main and test sources compiled. Full server integration tests, master runtime validation, and field validation were not run.
  • Reviewed SHA: 3002d3f6acca7eee6b8ea9ca9f2e158d65b93874.
  • Final local SHA: 3002d3f6acca7eee6b8ea9ca9f2e158d65b93874.
  • Confirmed remote branch and PR head SHA: 3002d3f6acca7eee6b8ea9ca9f2e158d65b93874.
  • Incomplete: Remote CI was not awaited; full server integration tests, master runtime validation, and field validation remain unrun.

@sonarqubecloud

sonarqubecloud Bot commented Nov 6, 2026

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant