Skip to content

[dev/1.3] Disable Subscription and Pipes containing data insertion under Ratis - #19157

Open
Caideyipi wants to merge 2 commits into
dev/1.3from
cp/19053-dev-1.3
Open

Caideyipi wants to merge 2 commits into
dev/1.3from
cp/19053-dev-1.3

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

Backport #19053 (ef9e799f50a522ff249f2f77a4a00fa413e46aa5) to dev/1.3, based on 82b5dcd52b208fd856e3648b6682d3b57e05ed95.

Disable Subscription and Pipes whose effective inclusion/exclusion parameters include data.insert when DataRegion consensus uses Ratis. Preserve metadata/deletion-only synchronization, recheck Pipe mutations inside procedure validation, discard disallowed recovered tasks, and reject insertion payloads at the receiver. Preserve the source fix for historical Ratis files with unknown Pipe progress.

Compatibility adaptations

  • Use the 1.3 extractor/connector accessors, tree-model metadata lookup, and existing receiver request types. Apply the historical-file change to PipeHistoricalDataRegionTsFileSource.
  • Keep messages in the existing bilingual ConfigNodeMessages and DataNodePipeMessages classes because 1.3 has no shared Pipe message source root.
  • Port the applicable regression scenarios and retain existing 1.3 tests. The source's table-model Pipe APIs, topic alteration, Subscription consensus queues/owner leases, and completed-empty-snapshot broker recovery have no 1.3 counterpart and are outside this backport.

Validation

  • Passed mvn clean test -P with-integration-tests -pl iotdb-api/pipe-api,iotdb-core/node-commons,iotdb-core/confignode,iotdb-core/datanode -am with the nine affected regression classes selected. All selected tests passed with no failures, errors, or skips.
  • Passed mvn clean test-compile -P with-integration-tests,with-zh-locale -pl iotdb-api/pipe-api,iotdb-core/node-commons,iotdb-core/confignode,iotdb-core/datanode -am -DskipTests.
  • Both commands ran on a231f2e89b978e87da69f50425b824157abb9f8e, rebuilding the required IoTDB module closure from clean source, using JDK 17 and a source-built TsFile dependency. Tests ran remotely; they were not run locally.
  • Spotless, git diff --check, preservation of existing target tests, bilingual message key/placeholder parity, and added-message checks passed.

The earlier full-reactor test command reached distribution but could not resolve reactor ZIP artifacts that the test phase does not produce. The focused module reactor above passed. Distribution packaging and integration tests were not run.

…er Ratis (#19053)

* [Bugfix] Replay Ratis historical TsFiles without Pipe progress

Treat MinimumProgressIndex as unknown for Ratis historical extraction so
successful writes remain eligible for replay after a leader change.
Preserve valid-index and other consensus-protocol coverage filtering.

Add a regression test for initial and resumed checkpoints and the
unaffected filtering behavior.

* Disable Pipe and Subscription with Ratis

* Restore ConfigNode descriptor imports

* Revert "Disable Pipe and Subscription with Ratis"

This reverts commit 7d4912d.

* Refine Ratis Pipe and Subscription restrictions

* Fix Ratis pipe recovery and receiver validation gaps

* Fix Pipe coordinator drop races and Ratis PlanNode admission

* Remove redundant consensus pipe exemptions from Ratis checks

* Address receiver review feedback

(cherry picked from commit ef9e799)
@Caideyipi
Caideyipi marked this pull request as draft November 6, 2026 08:16
@Caideyipi
Caideyipi marked this pull request as ready for review November 6, 2026 09:02
@sonarqubecloud

sonarqubecloud Bot commented Nov 6, 2026

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant