[Cosmos] Add Gateway service endpoint metric tags by jeet1995 · Pull Request #50888 · Azure/azure-sdk-for-java

Abhijeet Mohanty (jeet1995) · 2026-10-20T16:48:58Z

Motivation

Gateway request metrics currently lack a destination dimension, making it difficult to identify which regional endpoint is contributing failures, retries, or request charges. This also prevents separating traffic to Compute Gateway (443) from GatewayV2 (10250).

Approach

Extend the existing opt-in ServiceEndpoint dimension to Gateway requests. Attribute each recorded attempt to the destination it actually used, rather than assigning one endpoint to the entire logical operation: a single operation can retry across multiple destinations.

This lets users break down existing request metrics by endpoint without changing what those meters measure. Endpoint labels are reused across requests to avoid repeated formatting, and document paths are excluded so endpoint cardinality follows destinations rather than individual documents.

Compatibility

The dimension remains disabled by default and follows the existing tag-selection and suppression settings. Direct-mode metrics and operation-level measurements are unchanged.

Short workload check

Ran a local live point-read workload using the PR SDK at e76803a76ca and the benchmark's CsvMetricsReporter: 100 seeded documents (512-byte payload field), concurrency 4, 30 seconds per transport, CSV snapshots every 5 seconds. ServiceEndpoint was enabled and diagnostic-threshold filtering disabled. Fresh clients were used for each mode; response diagnostics verified the actual destination port and HTTP protocol.

Actual CsvMetricsReporter request-counter files and contents follow. Only the account name in the filenames is replaced with redacted-account; CSV timestamps and values are unchanged.

Compute Gateway

cosmos.client.req.gw.requests.RequestOperationType.Document_Read.RequestStatusCode.200_0.ServiceEndpoint.redacted-account-centralus.documents.azure.com_443.csv

t,count
2026-10-23T21:15:31.719476400Z,1
2026-10-23T21:15:36.761405200Z,144
2026-10-23T21:15:41.750171600Z,299
2026-10-23T21:15:46.751470300Z,458
2026-10-23T21:15:51.751075400Z,617
2026-10-23T21:15:56.754169400Z,777
2026-10-23T21:16:01.751206100Z,937
2026-10-23T21:16:01.830764100Z,941

GatewayV2

cosmos.client.req.gw.requests.RequestOperationType.Document_Read.RequestStatusCode.200_0.ServiceEndpoint.redacted-account-centralus.documents.azure.com_10250.csv

t,count
2026-10-23T21:16:03.864016500Z,1
2026-10-23T21:16:08.871347300Z,104
2026-10-23T21:16:13.872248300Z,262
2026-10-23T21:16:18.872340700Z,422
2026-10-23T21:16:23.872040400Z,579
2026-10-23T21:16:28.872130200Z,738
2026-10-23T21:16:33.872076Z,898
2026-10-23T21:16:33.899090400Z,902

All recorded reads succeeded (200/0). Each of the three request meters produced eight CSV snapshots per transport, with matching count deltas after excluding warm-up. The reporting registry omitted nonessential labels to keep CSV filenames short; endpoint, request type, and request status were retained. Temporary workload resources were deleted.

This checks live metric emission, not relative transport performance or the CPU/allocation overhead of tagging. Account names are redacted; regional suffixes are retained. No credentials or raw logs are included.

Sustained workload benchmark results

Benchmark table layout follows Azure/azure-sdk-for-java#48662. These are this PR's measurements, not values copied from that benchmark.

Coverage: 48 of 54 planned measured runs accepted. Gateway and GatewayV2 are complete at concurrency 8, 32, and 128; Direct is complete at 8 and 32. Direct concurrency 128 is incomplete: its first pair was stopped by the runner's five-minute required-metric gate while a CPU row was delayed in export (the row subsequently arrived). That partial pair and all smoke runs are excluded. No Direct-c128 performance result or full-matrix pass is claimed.

Test configuration

Baseline: published com.azure:azure-cosmos:4.83.0.
Feature: exact PR commit e76803a76ca9.
Harness: the same unchanged azure-cosmos-benchmark harness from that PR commit, with the same aligned runtime dependencies and JVM settings; only the Cosmos SDK artifact differs. Actual SDK class loading was verified. This evaluates 4.83.0 on the aligned dependency set, not its full published transitive-dependency stack.
Workload: single tenant, ReadThroughput, three independent JVM rounds per configuration, each with a recorded 600-second measurement window after seed/cool-down and 60-second read warm-up. Each process uses a native randomly seeded 1,000-item cohort with identical five-by-20-character data-field settings.
Workers: two matching Standard_D16s_v5 Linux VMs in Central US, baseline and feature assigned to separate fixed VMs, reached through Bastion. OpenJDK 21.0.12.1; G1GC; 8 GiB initial/max heap. The workers were not crossed over between SDKs, and endpoint routing was not pinned.
Telemetry: ServiceEndpoint explicitly enabled in both arms, preserving the other default tags; P50/P95/P99 configured. Diagnostic thresholds were set to zero for workload request-metric coverage. The native metrics and console exporters were enabled identically. Native Cosmos metric documents were ingested through Cosmos DB change feed into ADX.

Throughput and operation latency

Config	Concurrency	Connection mode	4.83.0 ops/s	PR ops/s	Paired delta	4.83.0 P99 ms	PR P99 ms	4.83.0 mean ms	PR mean ms
Read	8	Gateway / HTTP/1.1	2,498	2,575	+3.08%	7.01 [7.01-7.54]	7.01 [7.01-7.27]	3.17	3.06
Read	32	Gateway / HTTP/1.1	9,784	10,332	+5.61%	7.27 [7.27-7.54]	7.01	3.23	3.04
Read	128	Gateway / HTTP/1.1	38,381	39,444	+2.78%	7.01 [7.01-7.54]	7.01 [7.01-10.94]	3.26	3.17
Read	8	GatewayV2 / ThinClient / HTTP/2	3,030	2,934	-3.18%	6.23 [5.96-6.49]	6.23 [5.96-7.01]	2.60	2.68
Read	32	GatewayV2 / ThinClient / HTTP/2	11,580	12,133	+4.83%	6.49 [6.49-52.36]	6.49 [6.23-7.80]	2.72	2.59
Read	128	GatewayV2 / ThinClient / HTTP/2	38,329	39,017	+1.80%	6.75	6.75	3.15	3.08
Read	8	Direct / RNTBD	3,622	4,231	+16.89%	N/A	4.69 [4.69-4.95]	1.96	1.83
Read	32	Direct / RNTBD	14,351	14,330	-0.15%	5.47 [5.47-5.73]	5.21	2.19	2.03

Emit opt-in hostname and port dimensions for Gateway request metrics, with account-free coverage and a live integration test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

azure-pipelines · 2026-10-20T16:50:02Z

Azure Pipelines: Successfully started running 2 pipeline(s). 33 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run.

Carry immutable per-attempt URI references through response diagnostics instead of reparsing endpoint strings during metric emission. Preserve URI snapshots across errors and status remapping without changing diagnostic JSON. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Cache immutable endpoint Tags by request authority in the client endpoint manager and reuse them directly during metric emission. Avoid repeated formatting, escaping and Tag allocation on cache hits; clear cached tags when the endpoint manager closes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Own cached endpoint Tags in RxGatewayStoreModel and capture them per request attempt for success, failure and cancellation diagnostics. Keep metric emission independent of the document client and endpoint manager, preserve opt-in gating, and clear Gateway/thin-client caches during client shutdown. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…etric

Remove the added BridgeInternal overload, use existing diagnostics accessors for failure and cancellation recording, shorten metric documentation, and clean up imports and formatting. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Build fresh Gateway clients for Compute Gateway and HTTP/2 GatewayV2 coverage, assert actual transport and endpoint tags across CRUD and 404 responses, and wait for probe readiness. Use run-scoped database setup and unconditional cleanup without modifying the Direct metrics fixture. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Rename managementClient to resourceSetupClient to avoid confusion with Azure management SDKs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Abhijeet Mohanty (jeet1995) · 2026-10-20T21:37:17Z

/azp run java - cosmos - tests

azure-pipelines · 2026-10-20T21:37:53Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Extract telemetry eligibility checks, combine destination validation, and retain the explicit cache-hit path. Let the metrics producer own the NONE fallback and verify missing tags do not populate the cache. Document Gateway endpoint examples in the metric table. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Inject one throttled read on Compute Gateway and GatewayV2 and verify endpoint/status counter deltas match diagnostic attempts while the logical operation count increases once. Share probe-readiness setup with existing E2E cases. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

🟡 Changes recommended

Query-plan metrics have inconsistent tag keys, and Compute Gateway test assertions reject valid emulator ports.

2 open issues

What changed in this PR

Adds the opt-in ServiceEndpoint tag to Azure Cosmos DB Gateway request metrics, capturing each attempt’s destination while preserving existing defaults and Direct behavior.

Changes:

Cache destination tags and carry them through response and failure diagnostics.
Reuse captured tags during metric emission and clear caches on client shutdown.
Add unit and end-to-end coverage, metric documentation, and release notes.

File	Description
`sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/models/CosmosMetricTagName.java`	Documents Gateway endpoint values.
`sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/ThinClientStoreModel.java`	Passes telemetry configuration to the Gateway model.
`sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/RxGatewayStoreModel.java`	Caches endpoint tags and captures them per attempt.
`sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/RxDocumentClientImpl.java`	Wires configuration and cache cleanup.
`sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/directconnectivity/StoreResponseDiagnostics.java`	Snapshots destination URI and tag without changing JSON output.
`sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/directconnectivity/StoreResponse.java`	Preserves destination metadata during status remapping.
`sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/clienttelemetry/ClientTelemetryMetrics.java`	Emits captured endpoint tags with a `NONE` fallback.
`sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/ClientSideRequestStatistics.java`	Carries destination metadata into Gateway statistics.
`sdk/cosmos/azure-cosmos/docs/Metrics.md`	Adds Gateway endpoint examples.
`sdk/cosmos/azure-cosmos/CHANGELOG.md`	Announces the opt-in metric dimension.
`sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/implementation/RxGatewayStoreModelTest.java`	Tests capture on successful and failed responses.
`sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/GatewayEndpointMetricsTest.java`	Tests formatting, caching, suppression, and diagnostics.
`sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/GatewayEndpointMetricsE2ETest.java`	Exercises Gateway metrics for item operations and retries.

_{Review effort: Balanced}

Abhijeet Mohanty (jeet1995) · 2026-10-23T21:21:54Z

@sdkReviewAgent

Remove synthetic query-plan measurements and use actual endpoint-tagged request attempts, including retries and RU charges. Add schema and live fault-injection regressions, verify cached plans do not create network measurements, and accept captured Compute Gateway emulator ports. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Reuse DocumentCollection/QueryPlan for actual Gateway query-plan attempts while retaining endpoint attribution and retry accounting. Update compatibility assertions and release notes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Abhijeet Mohanty (jeet1995) · 2026-10-26T15:19:16Z

/azp run java - cosmos - tests

azure-pipelines · 2026-10-26T15:19:51Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Fabian Meiswinkel (FabianMeiswinkel)

LGTM

Add Gateway service endpoint metric tags

02408eb

Emit opt-in hostname and port dimensions for Gateway request metrics, with account-free coverage and a live integration test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions Bot added the Cosmos label Oct 20, 2026

Abhijeet Mohanty (jeet1995) and others added 8 commits October 20, 2026 13:07

Merge remote-tracking branch 'origin/main' into squad/http-endpoint-m…

6f9a1fc

…etric

Clarify Gateway metrics fixture resource client name

b04ceae

Rename managementClient to resourceSetupClient to avoid confusion with Azure management SDKs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Explain reuse of captured Gateway endpoint tags

f3cf4aa

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Abhijeet Mohanty (jeet1995) and others added 2 commits October 23, 2026 11:08

Abhijeet Mohanty (jeet1995) marked this pull request as ready for review October 23, 2026 20:57

Abhijeet Mohanty (jeet1995) requested review from a team and Kiran Kumar Kolli (kirankumarkolli) as code owners October 23, 2026 20:57

Copilot AI balanced review requested due to automatic review settings October 23, 2026 20:57

Copilot started reviewing on behalf of Abhijeet Mohanty (jeet1995) October 23, 2026 20:57

View session

Abhijeet Mohanty (jeet1995) changed the title ~~Add Gateway service endpoint metric tags~~ [Cosmos] Add Gateway service endpoint metric tags Oct 23, 2026

Copilot AI reviewed Oct 23, 2026

View reviewed changes

Comment thread sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/GatewayEndpointMetricsE2ETest.java Outdated

Comment thread ...os/src/main/java/com/azure/cosmos/implementation/clienttelemetry/ClientTelemetryMetrics.java

Abhijeet Mohanty (jeet1995) and others added 2 commits October 26, 2026 10:51

Fabian Meiswinkel (FabianMeiswinkel) approved these changes Oct 26, 2026

View reviewed changes

Abhijeet Mohanty (jeet1995) merged commit 54150b3 into Azure:main Oct 26, 2026
125 of 132 checks passed

Navigation Menu

Repository navigation

[Cosmos] Add Gateway service endpoint metric tags - #50888

[Cosmos] Add Gateway service endpoint metric tags#50888
Abhijeet Mohanty (jeet1995) merged 13 commits into
Azure:mainfrom
jeet1995:squad/http-endpoint-metric

Abhijeet Mohanty (jeet1995) commented Oct 20, 2026 •

edited

Loading

Uh oh!

azure-pipelines Bot commented Oct 20, 2026

Uh oh!

Abhijeet Mohanty (jeet1995) commented Oct 20, 2026

Uh oh!

azure-pipelines Bot commented Oct 20, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Abhijeet Mohanty (jeet1995) commented Oct 23, 2026

Uh oh!

Abhijeet Mohanty (jeet1995) commented Oct 26, 2026

Uh oh!

azure-pipelines Bot commented Oct 26, 2026

Uh oh!

Fabian Meiswinkel (FabianMeiswinkel) left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Abhijeet Mohanty (jeet1995) commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Approach

Compatibility

Short workload check

Compute Gateway

GatewayV2

Sustained workload benchmark results

Test configuration

Throughput and operation latency

Uh oh!

azure-pipelines Bot commented Oct 20, 2026

Uh oh!

Abhijeet Mohanty (jeet1995) commented Oct 20, 2026

Uh oh!

azure-pipelines Bot commented Oct 20, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Abhijeet Mohanty (jeet1995) commented Oct 23, 2026

Uh oh!

Abhijeet Mohanty (jeet1995) commented Oct 26, 2026

Uh oh!

azure-pipelines Bot commented Oct 26, 2026

Uh oh!

Fabian Meiswinkel (FabianMeiswinkel) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Abhijeet Mohanty (jeet1995) commented Oct 20, 2026 •

edited

Loading