You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Support asynchronous timeseries and table column deletion - #19186
Tree-model DELETE/DROP TIMESERIES and table-model ALTER TABLE ... DROP COLUMN now return after durable admission of a background procedure. Callers can inspect tree deletion with SHOW DELETE TIMESERIES PROGRESS and column deletion with DESCRIBE ... DETAILS, then wait for completion before dependent DDL.
Add a bounded, configurable tree-deletion scheduler that controls queue length, the minimum interval between execution starts, and task concurrency.
Preserve query-ID idempotency, expose task IDs and progress, and keep terminal progress separate from durable finalization. Return clear admission messages for asynchronous table and column deletion.
Persist admission and completion transitions, isolate stored procedure state, and retain same-database DDL reservations through database deletion/recreation and recovery. Keep tree/table database namespaces separate and protect database configuration from being overwritten during recreation.
Update integration-test helpers and dependent tests to wait for durable deletion completion, including JDBC and Session paths.
Tree-model asynchronous deletion configuration
Configure these ConfigNode settings in iotdb-system.properties; changes take effect after a ConfigNode restart.
Configuration
Default
Meaning
timeseries_deletion_queue_capacity
1024
Maximum number of tree-model timeseries deletion tasks waiting in the queue.
timeseries_deletion_execution_interval_ms
1000 (1 second)
Minimum interval between starting/dequeuing tree-model timeseries deletion tasks.
timeseries_deletion_max_concurrent_tasks
1
Maximum number of tree-model timeseries deletion tasks executing concurrently.
The execution interval applies when tasks leave the queue to start execution. Enqueue requests are not throttled by this interval. With the defaults, up to 1024 tasks can wait, one task executes at a time, and successive task starts are at least 1 second apart.
Example: submit deletions and track completion
CREATEDATABASEroot.async_delete_demo;
CREATE TIMESERIES root.async_delete_demo.d1.s1 WITH DATATYPE=DOUBLE, ENCODING=GORILLA;
CREATE TIMESERIES root.async_delete_demo.d1.s2 WITH DATATYPE=DOUBLE, ENCODING=GORILLA;
DELETE TIMESERIES root.async_delete_demo.d1.s1;
DELETE TIMESERIES root.async_delete_demo.d1.s2;
SHOW DELETE TIMESERIES PROGRESS;
Each DELETE TIMESERIES returns after its task has been durably admitted, with a task/procedure ID in the response message. It does not wait for deletion to finish. The two requests can be submitted back-to-back; with the default scheduler settings, their tasks execute one at a time and start at least 1 second apart.
SHOW DELETE TIMESERIES PROGRESS returns ProcedureId, QueryId, Paths, State, Progress, and ErrorMessage. Match each returned procedure ID and poll until its State is SUCCESS before recreating the corresponding timeseries or issuing dependent DDL. FINALIZING still means completion is being persisted; ROLLEDBACK indicates failure and should be checked with ErrorMessage.
For example, after the task for root.async_delete_demo.d1.s1 reports SUCCESS, the series can be recreated:
CREATE TIMESERIES root.async_delete_demo.d1.s1 WITH DATATYPE=DOUBLE, ENCODING=GORILLA;
Validation
Current submitted source: Spotless, git diff --check, and a clean English integration-test dependency-closure build passed (clean verify -pl integration-test -am -P with-integration-tests,SimpleIT). The selected audit IT was skipped by its existing class-level @Ignore (1 selected, 1 skipped); its test body did not run.
Immediately preceding snapshot, with identical production sources: 56 targeted unit tests and 8 targeted IT executions passed. The IT runs covered 1C1D and 1C3D, including Memory and PBTree schema modes for tree-model cases. The subsequent change only tightened the audit IT's procedure-ID completion wait; those runtime results are historical rather than reruns of the final snapshot.
Verification reused third-party dependency caches, disabled Maven extensions/build caching, and skipped Enforcer. Required IoTDB dependencies were built from clean source; downloaded IoTDB SNAPSHOT binaries were not used.
Not run: full-reactor English/Chinese builds, complete distribution packaging, the remaining affected IT matrix (including the Session-path test update), the audit IT body, and real-cluster leader-switch race scenarios.
This PR has:
been self-reviewed.
added unit tests or modified existing tests to cover new code paths.
added integration tests.
been tested in a test IoTDB cluster, with the snapshot and scope limits described above.
CI reported an error in IoTDBTableDDLHAIT.testHAWithOneDataNodeIsDown while the test waited for asynchronous timeseries deletion. The progress query used the cluster-wide result wrapper, which also queried the DataNode intentionally stopped by this test. The test now polls deletion progress through a single-node statement on its existing live connection.
Validation passed on Windows 11 with JDK 17: mvn -T 2 -Ddevelocity.off=true clean verify -DskipUTs -Dit.test=IoTDBTableDDLHAIT#testHAWithOneDataNodeIsDown -DfailIfNoTests=false -Dfailsafe.failIfNoSpecifiedTests=false -pl integration-test -am -PTableClusterIT -P with-integration-tests (1 test, 0 failures, 0 errors, 0 skipped). mvn spotless:apply -pl integration-test -P with-integration-tests and git diff --check also passed.
Updated from source head 3e66ecaf42e763ae42fc653dd27cf548fcbb9352; final commit: 971feafe81b9dec11a115f36cedfb8b715aa3aa2. Both the source branch and PR head match the final commit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Tree-model
DELETE/DROP TIMESERIESand table-modelALTER TABLE ... DROP COLUMNnow return after durable admission of a background procedure. Callers can inspect tree deletion withSHOW DELETE TIMESERIES PROGRESSand column deletion withDESCRIBE ... DETAILS, then wait for completion before dependent DDL.Tree-model asynchronous deletion configuration
Configure these ConfigNode settings in
iotdb-system.properties; changes take effect after a ConfigNode restart.timeseries_deletion_queue_capacity1024timeseries_deletion_execution_interval_ms1000(1 second)timeseries_deletion_max_concurrent_tasks1The execution interval applies when tasks leave the queue to start execution. Enqueue requests are not throttled by this interval. With the defaults, up to 1024 tasks can wait, one task executes at a time, and successive task starts are at least 1 second apart.
Example: submit deletions and track completion
Each
DELETE TIMESERIESreturns after its task has been durably admitted, with a task/procedure ID in the response message. It does not wait for deletion to finish. The two requests can be submitted back-to-back; with the default scheduler settings, their tasks execute one at a time and start at least 1 second apart.SHOW DELETE TIMESERIES PROGRESSreturnsProcedureId,QueryId,Paths,State,Progress, andErrorMessage. Match each returned procedure ID and poll until itsStateisSUCCESSbefore recreating the corresponding timeseries or issuing dependent DDL.FINALIZINGstill means completion is being persisted;ROLLEDBACKindicates failure and should be checked withErrorMessage.For example, after the task for
root.async_delete_demo.d1.s1reportsSUCCESS, the series can be recreated:Validation
git diff --check, and a clean English integration-test dependency-closure build passed (clean verify -pl integration-test -am -P with-integration-tests,SimpleIT). The selected audit IT was skipped by its existing class-level@Ignore(1 selected, 1 skipped); its test body did not run.This PR has:
Key changed classes
ProcedureManager,ProcedureExecutor,DeleteTimeSeriesProcedure,DeleteTimeSeriesProcedureScheduler,ConfigProcedureStore, SQL parsing/progress RPC classes, and integration-test completion helpers.