Skip to content

Expose internal client sessions through multiplexed RPC - #19125

Open
shuwenwei wants to merge 33 commits into
masterfrom
feature/internal-rpc-inherit-client-rpc
Open

shuwenwei wants to merge 33 commits into
masterfrom
feature/internal-rpc-inherit-client-rpc

Conversation

@shuwenwei

@shuwenwei shuwenwei commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

Description

Allow trusted internal callers to reuse the client session protocol on the existing DataNode internal endpoint without changing the legacy internal RPC protocol.

RPC routing and compatibility

  • Keep IDataNodeRPCService as an independent service and preserve all existing internal RPC method names and semantics.
  • Route unprefixed requests to the default IDataNodeRPCService processor.
  • Register IClientRPCService under the service name prefix IClientRPCService: on the same dn_internal_port.
  • Preserve mixed-version behavior for existing ConfigNode/DataNode internal calls.
dn_internal_port
└── TMultiplexedProcessor
    ├── default / no prefix
    │   └── IDataNodeRPCService.Processor
    │       └── DataNodeInternalRPCServiceImpl
    └── "IClientRPCService:<method>"
        └── IClientRPCService.Processor
            └── InternalClientRPCServiceImpl

On the client side, TBinaryProtocol or TCompactProtocol is wrapped by TMultiplexedProtocol for internal sessions. The service name is retained across reconnects.

Internal sessions

  • Add IInternalSessionBuilder<T> and buildInternal(INodeSupplier) entry points for tree sessions, table sessions, table pools, subscription-management sessions, and table pull consumers.
  • Propagate the service name through initial connections and reconnects.
  • Disable discovery and redirection for internal sessions and keep the caller-owned INodeSupplier lifecycle outside the session.
  • Add SSL configuration propagation for subscription session and consumer clients.

Authentication and trust boundary

Internal client RPC is used for communication between trusted internal nodes through the DataNode internal endpoint. It is separate from the client-facing RPC service and relies on operator-controlled network isolation for the cluster internal network.

The internal login path therefore trusts the username supplied by the internal caller and does not require a password. It still validates that the user exists and applies the configured account login policy. Existing operation-level authorization, quota, and query-ownership checks remain in effect. External clients continue to use the normal password/TOTP authentication flow on the client RPC endpoint.

Session lifecycle

  • Add a server-side passwordless login path only for the multiplexed internal client RPC service.
  • Register client sessions only when a request enters the multiplexed client processor.
  • Run full connection cleanup after session invalidation and before removing the ThreadLocal session.

Validation

  • git diff --check passed.
  • Compilation, formatter, unit tests, integration tests, and cluster validation were not run in this environment.

@codecov

codecov Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 35.34202% with 397 lines in your changes missing coverage. Please review.
✅ Project coverage is 48.75%. Comparing base (9a386ab) to head (e76cc51).

Files with missing lines Patch % Lines
...ache/iotdb/db/protocol/session/SessionManager.java 0.00% 70 Missing ⚠️
...n/consumer/base/AbstractSubscriptionProviders.java 0.00% 68 Missing ⚠️
...db/db/queryengine/plan/analyze/AnalyzeVisitor.java 31.81% 30 Missing ⚠️
...on/consumer/base/AbstractSubscriptionConsumer.java 0.00% 23 Missing ⚠️
...umer/base/AbstractSubscriptionConsumerBuilder.java 0.00% 14 Missing ⚠️
.../subscription/SubscriptionTableSessionBuilder.java 7.14% 13 Missing ⚠️
...er/table/SubscriptionTablePullConsumerBuilder.java 0.00% 13 Missing ⚠️
...n/subscription/SubscriptionTreeSessionBuilder.java 0.00% 12 Missing ⚠️
...er/table/SubscriptionTablePushConsumerBuilder.java 0.00% 12 Missing ⚠️
...umer/tree/SubscriptionTreePullConsumerBuilder.java 0.00% 12 Missing ⚠️
... and 29 more
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #19125      +/-   ##
============================================
- Coverage     48.76%   48.75%   -0.01%     
- Complexity      674      689      +15     
============================================
  Files          5574     5588      +14     
  Lines        412340   412780     +440     
  Branches      54379    54406      +27     
============================================
+ Hits         201068   201243     +175     
- Misses       211272   211537     +265     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Revert the prior approach that exposed the complete client RPC surface by inheritance and renamed internal RPC methods.

The replacement will keep IDataNodeRPCService stable and multiplex the client service on the existing internal endpoint.
Preserve IDataNodeRPCService as the default internal processor and expose IClientRPCService through a service-name prefix on the same endpoint.

Add internal session, table pool, subscription, reconnect, and SSL support with passwordless internal login constrained by account policy.
@shuwenwei shuwenwei changed the title Support internal sessions through inherited client RPC services Expose internal client sessions through multiplexed RPC Nov 5, 2026
Verify multiplexed and default processor routing, session reconnects, subscription SSL forwarding, and an internal-session integration path.
# Conflicts:
#	iotdb-client/session/src/main/java/org/apache/iotdb/session/Session.java
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/analyze/load/LoadTsFileAnalyzer.java
@sonarqubecloud

sonarqubecloud Bot commented Nov 6, 2026

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant