Skip to content

Support flow-controlled gRPC client requests - #3430

Open
wasphin wants to merge 1 commit into
apache:masterfrom
wasphin:feature/grpc-client-h2-flow-control
Open

Support flow-controlled gRPC client requests#3430
wasphin wants to merge 1 commit into
apache:masterfrom
wasphin:feature/grpc-client-h2-flow-control

Conversation

@wasphin

@wasphin wasphin commented Aug 9, 2026

Copy link
Copy Markdown
Member

What problem does this PR solve?

Issue Number: #1346, #1770, #1983, #2117

Problem Summary:

The gRPC client previously attempted to serialize the entire request into
HTTP/2 DATA frames immediately.

When the request size exceeded the peer's available connection-level or
stream-level flow-control window, frame generation failed and the RPC returned
an error instead of waiting for WINDOW_UPDATE.

The client needs to retain unsent DATA while waiting for the peer to restore
the flow-control window. This retained DATA must be bounded and released with
the RPC lifecycle to prevent excessive memory usage when the peer does not send
WINDOW_UPDATE.

What is changed and the side effects?

Changed:

  • Split client request DATA frames according to both connection-level and
    stream-level HTTP/2 flow-control windows.

  • Store DATA that cannot be sent immediately in H2StreamContext.

  • Resume sending pending DATA when a valid WINDOW_UPDATE restores the
    available window.

  • Track the total pending request DATA size for each H2 connection.

  • Use socket_max_unwritten_bytes as the upper bound for pending request DATA.

  • Reject new requests with EOVERCROWDED after a connection reaches the
    pending DATA limit.

Side effects:

  • Performance effects:
    • Requests blocked by remote flow control remain buffered until the peer
      sends WINDOW_UPDATE, the RPC finishes, or the stream is removed.
    • Pending DATA accounting and flow-control operations use the existing H2
      stream mutex.
    • Fragmented requests may require additional DATA frame serialization and
      socket writes.
  • Breaking backward compatibility: N/A

Check List:

- Split client request DATA frames according to the peer's connection and
  stream flow-control windows.
- Buffer unsent DATA and resume transmission when WINDOW_UPDATE restores
  capacity.
- Track pending request bytes per H2 connection and apply
  socket_max_unwritten_bytes as an upper bound.
- Reject or reroute new requests once the pending DATA limit is reached.
- Release buffered DATA when an RPC fails, times out, or its stream is
  removed.
- Add tests for fragmented transmission, deferred DATA flushing,
  pending-byte accounting, and buffer cleanup.
@wasphin
wasphin force-pushed the feature/grpc-client-h2-flow-control branch from d0d8292 to e9850e1 Compare August 9, 2026 15:54
@wwbmmm
wwbmmm requested a lite review from Copilot August 10, 2026 02:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds client-side HTTP/2 (gRPC) request flow-control compliance in bRPC by buffering unsent request DATA when remote connection/stream windows are exhausted, then resuming transmission on WINDOW_UPDATE, with a per-connection pending-data cap to bound memory usage.

Changes:

  • Buffer unsent HTTP/2 request DATA per stream and flush it when WINDOW_UPDATE/SETTINGS increase the usable window.
  • Track per-connection total pending request DATA and reject new requests with EOVERCROWDED once the configured cap is reached.
  • Extend/adjust unit tests to validate request buffering, DATA splitting, and window-update behavior.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
test/brpc_http_rpc_protocol_unittest.cpp Updates HTTP/2 flow-control test to validate buffering/cleanup behavior.
test/brpc_h2_unsent_message_unittest.cpp Adds new unit tests for DATA splitting, buffering, and pending-data limit behavior.
test/brpc_grpc_protocol_unittest.cpp Adds regression test for sending large gRPC requests with small remote flow-control windows.
src/brpc/policy/http2_rpc_protocol.h Introduces per-stream pending request DATA and per-connection pending-data accounting APIs.
src/brpc/policy/http2_rpc_protocol.cpp Implements buffering/flush logic, SETTINGS/WINDOW_UPDATE integration, and pending-data limit checks.
Suppressed comments (2)

test/brpc_h2_unsent_message_unittest.cpp:59

  • PopFrame() passes &(*payload)[0] to cutn() even when frame.payload_size == 0. Taking operator[] on an empty std::string is undefined behavior and can trip sanitizers if a 0-length frame is ever parsed in this test.
    payload->resize(frame.payload_size);
    CHECK_EQ(frame.payload_size,
             buf->cutn(&(*payload)[0], frame.payload_size));
    return frame;

test/brpc_grpc_protocol_unittest.cpp:298

  • After switching to an ephemeral port, initialize the client channel using server.listen_address() to avoid hard-coding the address/port.
    brpc::Channel channel;
    brpc::ChannelOptions channel_options;
    channel_options.protocol = g_protocol;
    channel_options.timeout_ms = 10000;
    ASSERT_EQ(0, channel.Init("127.0.0.1:8012", "", &channel_options));

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines 24 to 28
#include "bthread/bthread.h"
#include "butil/atomicops.h"
#include "brpc/policy/http_rpc_protocol.h"
#include "brpc/policy/http2_rpc_protocol.h"
#include "gperftools_helper.h"
Comment on lines +1561 to +1564
ASSERT_GT(ctx->_pending_data_size, 0u);
h2_req->DestroyStreamUserData(
_h2_client_sock, &cntl, ECANCELED, false);
ASSERT_EQ(0u, ctx->_pending_data_size);
Comment on lines +290 to +293
brpc::ServerOptions server_options;
server_options.h2_settings.stream_window_size = 32;
ASSERT_EQ(0, server.Start("127.0.0.1:8012", &server_options));

Comment on lines +1648 to +1651
if (ctx->PendingDataOvercrowded()) {
return butil::Status(EOVERCROWDED,
"Too much pending HTTP/2 request data");
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants