[1.3] Backport fix duplicate scheduling in procedure execution (#17902)#6
Open
Caideyipi wants to merge 102 commits into
Open
[1.3] Backport fix duplicate scheduling in procedure execution (#17902)#6Caideyipi wants to merge 102 commits into
Caideyipi wants to merge 102 commits into
Conversation
…#17464) (apache#17518) * fix tsblock deserialize * fix ut error on win * Revert "fix ut error on win" This reverts commit 34b8de4.
… bug of third-party subscription model (apache#17524) * complete * Debounce
…le parser (apache#17504) (apache#17523) * [To dev/1.3] Pipe: Upgraded bouncy castle && Added logs for pipe tsFile parser (apache#17504) * fixed
…available memory in the memoryPool is insufficient (apache#17529)
…ration only to target nodes and harden compaction schedule interruption handling (apache#17447)
…t frame is oversize (apache#17550)
…ransfer historical data when it is altered to a historical pipe (apache#17223) (apache#17226)
…d in temporary timeout exception & Optimized the directory check in receiver & Optimized the configNode pipe logic (apache#17556) & Optimized the clear logic of Schema Region (apache#17553) (apache#17576) * Optimized the clear logic of Schema Region && Pipe: Fixed the bug that the historical pipe does not work for deletion-only sync (apache#17553) * Pipe: Fixed the bug that air gap receiver may not respond in temporary timeout exception & Optimized the directory check in receiver & Optimized the configNode pipe logic (apache#17556)
…erence & will wait long time after pipe close (apache#17549) (apache#17569) * Pipe: Implementing DisruptorQueue (apache#16639) * Pipe: Fixed the bug that Disruptor may not clear the reference & will wait long time after pipe close (apache#17549) * fix * fix * del --------- Co-authored-by: Zhenyu Luo <zhenyu@apache.org>
…nd PipeRemainingOperator (apache#17584) (apache#17602)
…size (apache#17603) * Add thrift max frame size calculate logic * Add thrift max frame size calculate logic * fix review
… but encountered sync failure later (apache#17586) (apache#17605) * fix * fix
…upted or ring buffer full (apache#17673) (apache#17818) * Fix interrupt * spotless (cherry picked from commit b614d6d)
…pache#17820) * fix * sp * Fix CQ local schedule cancellation
…eratorTest (apache#17664) (apache#17819) * Opt * Update UnclosedFileScanHandleImpl.java * Update StorageEngine.java * Update ClosedFileScanHandleImpl.java * column index * spt * Address performance review comments * fix (cherry picked from commit 89730b1)
…17734) (apache#17820)" (apache#17826) This reverts commit 24be79a.
…#17734) (apache#17820)" (apache#17826) (apache#17829) This reverts commit 5e1587f.
…) (apache#17817) * C++ client: add thread-safe SessionPool, enable RPC compression, and harden buffers (apache#17800) * Wire RPC compression flag through Session to its connections The enableRPCCompression option set via Session::open(bool) or the session builder was never propagated to SessionConnection, whose flag was hardcoded to false, so the compact Thrift protocol never took effect. Thread the flag from the builder/open() into both the data SessionConnection and the node-discovery NodesSupplier client so compression actually applies. * Use snprintf for Tablet bounds-check error messages Tablet::addValue and the OBJECT-value overload formatted out-of-range diagnostics with sprintf into a fixed 100-byte stack buffer, risking an overflow. Switch to snprintf bounded by sizeof(buffer) and cast the size_t arguments to long to match the %ld format. * Append big-endian bytes in MyStringBuffer instead of overwriting On big-endian hosts MyStringBuffer::putOrderedByte used str.assign, which replaced the whole buffer with each numeric write and corrupted previously serialized content. Use str.append so bytes accumulate, matching the little-endian path. * Add thread-safe SessionPool to the C++ client Introduce SessionPool and SessionPoolBuilder so multiple threads can share a bounded set of connections without external locking. A single Session is not safe to use concurrently, so the pool lends each Session to one borrower at a time via an RAII PooledSession handle and reclaims it on scope exit. Sessions are created outside the lock to avoid blocking other borrowers during the handshake, and getSession() blocks up to a configurable timeout when the pool is exhausted. Query results are returned as a PooledSessionDataSet that keeps the Session leased until the result set is fully read, since SessionDataSet lazily fetches further blocks over the same connection. Connections that raise IoTDBConnectionException are evicted rather than recycled. Add integration tests covering basic borrow/insert/query, concurrent writers, and exhaustion-timeout behavior. * Reject zero maxSize in SessionPool instead of clamping to 1 Address review feedback: maxSize is size_t, so a non-positive check reduces to == 0 (and "<= 0" would be a tautological-comparison warning under -Wall). Rather than silently clamping an invalid 0 to 1, fail fast by throwing IoTDBException so the misuse surfaces at construction time. * Tolerate missing timeseries in SessionPool test cleanup The pre-test cleanup deleted root.test.pool.* timeseries unconditionally, which threw 508 (does not exist) on a fresh database and failed the new [sessionPool] cases. Ignore that error since the cleanup is best-effort. * Revert "Wire RPC compression flag through Session to its connections" This reverts commit 2f35cc5. Honoring the compression flag makes the client negotiate the compact Thrift protocol, which the binary-only IoTDB server used by the C++ integration tests cannot speak, breaking the pre-existing ts_session_open_with_compression smoke test (it had only passed because the flag was a no-op). Compression needs a compact-protocol-enabled test server, so it will be reintroduced in a dedicated PR with the matching server-side test support. SessionPool keeps its compression option for forward compatibility; it is currently a no-op, as the rest of the client has always been. * fix format * Discard SessionPool session if pool closed during construction Address review feedback: acquire() releases the lock while building a new connection, so a concurrent close() could set closed_ after the slot was reserved, and the freshly opened session would still be handed out from a closed pool. Re-check closed_ under the lock after construction; if the pool was closed meanwhile, release the slot, tear the session down outside the lock, and throw instead of returning it. * C++ client: adapt SessionPool to dev/1.3 Session/builder APIs The cherry-picked SessionPool targeted master's expanded AbstractSessionBuilder (DEFAULT_* constants plus nodeUrls/connectTimeoutMs/useSSL/trustCertFilePath fields), none of which exist on dev/1.3, so the C++ client failed to compile. Adapt only the newly added SessionPool.h/.cpp (no existing dev/1.3 interface or implementation is changed): - Replace AbstractSessionBuilder::DEFAULT_* with dev/1.3's literal defaults. - Keep connectTimeoutMs (still applied via Session::open()). - Drop the multi-node (nodeUrls) constructor and SSL options, since dev/1.3's Session(AbstractSessionBuilder*) wires neither and would silently ignore them.
(cherry picked from commit 99f0af1)
… printing JVM crash (apache#17675) (apache#17830)
…17807) (apache#17833) * Pipe: account page decode memory in scan parser (apache#17807) * Pipe: account page decode memory in scan parser * Fix pipe scan parser single page row count * Fix pipe scan parser page memory test * Fix tsfile writer API in pipe test
… the same jar (apache#17732) (apache#17835) * UDF Fix * sp * fix * Filter invalid jar reference counts in snapshots
(cherry picked from commit 2fd70b9)
…pache#17848) (cherry picked from commit eaa5bcb)
* Fix count and show timeseries follow-up (apache#17804) * Fix count and show timeseries follow-up * Refine internal timeseries count permissions * fix * Fix timeseries count source permission tests
* Pipe: Fixed the bug that separated historical pipe may not include mod on deletion & The pipe without data.insert may be wrongly separated by pipe and transfer data (apache#17346) * fix * fix * f (cherry picked from commit 355a872) * Pipe: Made the historical pipe split auto dropped after completion (apache#17295) * snapshot * may-comp * auto (cherry picked from commit 9bfe0b0)
) Backport receiver-side tree-model changes to dev/1.3. The 2.x table-model load analyzer changes and table/dual ITs are not applicable to dev/1.3. (cherry picked from commit 90055d5)
…he#17883) * Support sink config key for pipe request slicing * Support processor output series aliases
* Fix Windows reserved device name validation * Reject additional illegal Windows path segments * Reject empty directory path segments (cherry picked from commit 3d98ea5)
* [to dev/1.3] Fix tree model WAL serialized sizes * Fix WAL charset deserialization * Fix insert node serde charset handling * spotless
* refactor(client-cpp): port SDK refactor to dev/1.3 without table model Reorganize the C++ client into include/session/rpc with CMake-driven build, embedded Thrift, Windows DLL packaging, SessionC API, and CI release workflows. Scope excludes 2.x table-model APIs and Thrift fields. * fix(client-cpp): align cmake plugin and clang-format for CI Replace pluginManagement cmake-maven-plugin with io.github 4.2.3-b1 so with-cpp Maven builds resolve the plugin. Pin clang.format.version to 17.0.6 and apply Spotless formatting to C++ sources. * fix(ci): pin clang-format 17 in multi-language-client cpp job ubuntu-latest ships clang-format 18.x which breaks Spotless when clang.format.version is 17.0.6. Install LLVM 17 clang-format on Linux, macOS, and Windows like refactor/cpp-client-sdk. * fix(build): bump Maven wrapper to 3.9.12 for cmake-maven-plugin cmake-maven-plugin 4.2.3-b1 requires Maven 3.9.9+. The wrapper was on 3.9.6, causing with-cpp CI to fail even when using ./mvnw. * fix(client-cpp): include vector in AbstractSessionBuilder.h GCC on Linux CI fails when Session.h includes AbstractSessionBuilder.h before vector; add missing #include <vector> for nodeUrls member. * fix(client-cpp): fix query result parsing and IT on IoTDB 1.3 Apply server columnNameIndexMap for TsBlock reads, align Time column handling with the Java client, and read text types by physical column type. Defer C API session open, disable auto-fetch in tests, and fall back when SHOW AVAILABLE URLS is unsupported. * fix(client-cpp): address PR review items for public API and Catch2 - Revert unrelated .gitignore changes (.run, .claude, relational-grammar) - Remove using namespace std from public headers Session.h and Common.h - Download Catch2 at build time via Maven/CMake instead of vendoring catch.hpp * style(client-cpp): apply clang-format to Session.h * fix(client-cpp): download Catch2 before CMake compile phase Run wget in generate-resources (not generate-test-resources, which runs after compile). CMake also downloads catch.hpp when the header is still missing. * fix(client-cpp): qualify std types for MSVC after removing using namespace std - SessionConnection.h/cpp: std::string, std::vector, std::exception, std::shared_ptr - IoTDBRpcDataSet.cpp: std::exception in catch handlers - Session.h: (std::max) to avoid Windows max macro conflict * fix(client-cpp): use sbin start/stop scripts on Windows Align Windows profile paths with the 1.3 distribution layout (scripts under sbin/, not windows/). * refactor(client-cpp): remove table-model docs and dead code from dev/1.3 port Align example READMEs with the tree-only scope, drop unused table-model API surface, and remove IDeviceID redirect helpers that have no callers. * fix(ci): auto-detect Visual Studio generator for cpp on Windows windows-latest now ships VS 2026 while pom.xml defaults to VS 2022. Use vswhere in the multi-language-client workflow and pass -Dcmake.generator so CMake matches the installed toolchain. * fix(ci): align cpp Windows matrix with refactor/cpp-client-sdk workflow Replace windows-latest and broken vswhere detection with explicit windows-2022 and windows-2025-vs2026 runners; pass Visual Studio 18 2026 generator only on the VS 2026 image. Also match ubuntu/mac matrix and add Spotless check from the refactor branch. * Sync C++ client packaging updates to 1.3 * Fix VS2017 C++ package Boost setup * Fix Windows C++ client CI dependencies * Fix C++ client workflow verification
…pache#17943) (cherry picked from commit 8fd6b04)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of 0a45a3b to dev/1.3.\n\nFocused verification: mvn -pl iotdb-core/confignode -DskipITs -Dspotless.check.skip=true -Dcheckstyle.skip=true -Dtest=TestProcedureExecutor,TestLockRegime,PipeTaskCoordinatorLockTest,PipeHandleLeaderChangeProcedureTest test