Skip to content

Add code coverage report flags#11981

Open
juan-fernandez wants to merge 2 commits into
masterfrom
juan-fernandez/coverage-report-flags
Open

Add code coverage report flags#11981
juan-fernandez wants to merge 2 commits into
masterfrom
juan-fernandez/coverage-report-flags

Conversation

@juan-fernandez

Copy link
Copy Markdown
Contributor

What Does This Do

Adds support for DD_CODE_COVERAGE_FLAGS to Test Optimization coverage-report uploads. Values are parsed as comma-separated flags, trimmed, and included in the multipart event as the JSON string array report.flags.

The configuration preserves flag order and duplicates, removes empty entries, and applies the same immutable snapshot to every report uploaded by the process. More than 32 flags logs a warning, omits report.flags, and continues the upload.

Registers the configuration metadata and migrates CoverageReportUploaderTest from Spock/Groovy to JUnit 5 Java.

Motivation

Coverage reports uploaded by the Java tracer could not use the same grouping flags supported by datadog-ci coverage upload --flags. This makes the Java uploader compatible with that payload shape while preserving fail-open tracer behavior.

Additional Notes

Validation:

  • ./gradlew :internal-api:test --tests datadog.trace.api.ConfigCodeCoverageFlagsTest
  • ./gradlew :dd-java-agent:agent-ci-visibility:test --tests datadog.trace.civisibility.coverage.report.CoverageReportUploaderTest
  • ./gradlew :internal-api:test --tests datadog.trace.api.telemetry.ConfigInversionMetricCollectorImplTest
  • ./gradlew :dd-trace-api:spotlessCheck :internal-api:spotlessCheck :dd-java-agent:agent-ci-visibility:spotlessCheck
  • ./gradlew :internal-api:compileJava

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@juan-fernandez juan-fernandez added comp: ci visibility Continuous Integration Visibility type: feature Enhancements and improvements tag: ai generated Largely based on code generated by an AI or LLM labels Jul 17, 2026 — with ChatGPT Codex Connector
@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 17, 2026

Copy link
Copy Markdown

Test Environment - sbt-scalatest

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 55.15 55.43 $\color{green}{\blacktriangledown}$ -0.28 54.33 $\color{red}{\blacktriangle}$ +0.82 68/240
agentEvpProxy 54.73 n/a n/a n/a n/a -

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 17, 2026

Copy link
Copy Markdown

Test Environment - nebula-release-plugin

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 36.53 36.42 $\color{red}{\blacktriangle}$ +0.11 36.42 $\color{red}{\blacktriangle}$ +0.11 37/120
agentless 35.09 36.42 $\color{green}{\blacktriangledown}$ -1.33 36.42 $\color{green}{\blacktriangledown}$ -1.33 37/120
agentlessCodeCoverage 44.74 44.48 $\color{red}{\blacktriangle}$ +0.26 44.48 $\color{red}{\blacktriangle}$ +0.26 37/120
agentlessLineCoverage 74.84 74.82 $\color{red}{\blacktriangle}$ +0.02 74.82 $\color{red}{\blacktriangle}$ +0.02 37/120

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@datadog-prod-us1-3

datadog-prod-us1-3 Bot commented Jul 17, 2026

Copy link
Copy Markdown

🎯 Code Coverage (details)
Patch Coverage: 23.81%
Overall Coverage: 57.25% (-0.05%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 34c0678 | Docs | Datadog PR Page | Give us feedback!

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 17, 2026

Copy link
Copy Markdown

Test Environment - netflix-zuul

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 88.43 87.80 $\color{red}{\blacktriangle}$ +0.63 87.80 $\color{red}{\blacktriangle}$ +0.63 37/128
agentless 81.98 81.05 $\color{red}{\blacktriangle}$ +0.93 81.05 $\color{red}{\blacktriangle}$ +0.93 36/126
agentlessCodeCoverage 96.99 97.04 $\color{green}{\blacktriangledown}$ -0.05 95.12 $\color{red}{\blacktriangle}$ +1.87 36/124
agentlessLineCoverage 113.56 111.62 $\color{red}{\blacktriangle}$ +1.94 111.62 $\color{red}{\blacktriangle}$ +1.94 35/123

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 17, 2026

Copy link
Copy Markdown

Test Environment - pass4s

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 4.95 9.35 $\color{green}{\blacktriangledown}$ -4.40 10.33 $\color{green}{\blacktriangledown}$ -5.38 31/113
agentless 9.71 10.33 $\color{green}{\blacktriangledown}$ -0.62 10.13 $\color{green}{\blacktriangledown}$ -0.42 31/113
agentlessCodeCoverage 15.52 17.03 $\color{green}{\blacktriangledown}$ -1.51 17.38 $\color{green}{\blacktriangledown}$ -1.86 30/111

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 17, 2026

Copy link
Copy Markdown

Test Environment - reactive-streams-jvm

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 22.59 22.09 $\color{red}{\blacktriangle}$ +0.50 21.65 $\color{red}{\blacktriangle}$ +0.94 42/131
agentless 18.80 18.82 $\color{green}{\blacktriangledown}$ -0.02 18.82 $\color{green}{\blacktriangledown}$ -0.02 40/127
agentlessCodeCoverage 20.13 20.39 $\color{green}{\blacktriangledown}$ -0.26 19.99 $\color{red}{\blacktriangle}$ +0.14 38/124
agentlessLineCoverage 30.63 30.42 $\color{red}{\blacktriangle}$ +0.21 29.82 $\color{red}{\blacktriangle}$ +0.81 37/123

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 17, 2026

Copy link
Copy Markdown

Test Environment - heliboard

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 9.83 9.35 $\color{red}{\blacktriangle}$ +0.48 9.54 $\color{red}{\blacktriangle}$ +0.29 36/51

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@dd-octo-sts

dd-octo-sts Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.96 s 14.03 s [-1.3%; +0.2%] (no difference)
startup:insecure-bank:tracing:Agent 13.02 s 13.00 s [-0.6%; +1.0%] (no difference)
startup:petclinic:appsec:Agent 16.89 s 16.75 s [-0.1%; +1.9%] (no difference)
startup:petclinic:iast:Agent 16.85 s 16.92 s [-1.1%; +0.3%] (no difference)
startup:petclinic:profiling:Agent 16.72 s 16.76 s [-1.6%; +1.0%] (no difference)
startup:petclinic:sca:Agent 16.39 s 16.76 s [-6.7%; +2.3%] (no difference)
startup:petclinic:tracing:Agent 15.99 s 16.15 s [-2.0%; -0.0%] (maybe better)

Commit: 34c06786 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 17, 2026

Copy link
Copy Markdown

Test Environment - sonar-kotlin

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 11.93 12.87 $\color{green}{\blacktriangledown}$ -0.94 12.87 $\color{green}{\blacktriangledown}$ -0.94 35/120
agentless 11.00 11.88 $\color{green}{\blacktriangledown}$ -0.88 12.12 $\color{green}{\blacktriangledown}$ -1.12 33/117
agentlessCodeCoverage 14.22 14.51 $\color{green}{\blacktriangledown}$ -0.29 15.11 $\color{green}{\blacktriangledown}$ -0.89 33/117
agentlessLineCoverage 18.26 18.82 $\color{green}{\blacktriangledown}$ -0.56 18.82 $\color{green}{\blacktriangledown}$ -0.56 34/119

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@datadog-prod-us1-3

datadog-prod-us1-3 Bot commented Jul 17, 2026

Copy link
Copy Markdown

Bits couldn’t fix the CI

🟢 Investigated · 🔴 Fix failed · ⚪ Validate · ⚪ Ready

validate_supported_configurations_v2_local_file fails because DD_CODE_COVERAGE_FLAGS has no implementation in the external configuration registry. The local metadata and code declaration are consistent; adding the configuration version to that registry requires external registry access, so no code change was made.


View in Datadog | Reviewed commit f6fd6c1

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 17, 2026

Copy link
Copy Markdown

Test Environment - jolokia

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 94.06 93.23 $\color{red}{\blacktriangle}$ +0.83 93.23 $\color{red}{\blacktriangle}$ +0.83 42/133
agentless 88.82 89.58 $\color{green}{\blacktriangledown}$ -0.76 89.58 $\color{green}{\blacktriangledown}$ -0.76 43/131
agentlessCodeCoverage 98.22 99.00 $\color{green}{\blacktriangledown}$ -0.78 99.00 $\color{green}{\blacktriangledown}$ -0.78 42/128
agentlessLineCoverage 99.64 99.00 $\color{red}{\blacktriangle}$ +0.64 99.00 $\color{red}{\blacktriangle}$ +0.64 40/126

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 17, 2026

Copy link
Copy Markdown

Test Environment - okhttp

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 19.50 19.20 $\color{red}{\blacktriangle}$ +0.30 19.20 $\color{red}{\blacktriangle}$ +0.30 35/127
agentless 20.64 18.45 $\color{red}{\blacktriangle}$ +2.19 18.82 $\color{red}{\blacktriangle}$ +1.82 33/124
agentlessCodeCoverage 23.08 22.09 $\color{red}{\blacktriangle}$ +0.99 22.09 $\color{red}{\blacktriangle}$ +0.99 34/123
agentlessLineCoverage 46.21 44.48 $\color{red}{\blacktriangle}$ +1.73 44.48 $\color{red}{\blacktriangle}$ +1.73 33/129

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@daniel-mohedano daniel-mohedano left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, left a small nit 🚀

Comment thread internal-api/src/main/java/datadog/trace/api/Config.java Outdated
@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 17, 2026

Copy link
Copy Markdown

Test Environment - spring_boot

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 16.35 16.04 $\color{red}{\blacktriangle}$ +0.31 16.04 $\color{red}{\blacktriangle}$ +0.31 35/117
agentless 9.77 9.73 $\color{red}{\blacktriangle}$ +0.04 9.73 $\color{red}{\blacktriangle}$ +0.04 35/118
agentlessCodeCoverage 13.87 13.40 $\color{red}{\blacktriangle}$ +0.47 13.40 $\color{red}{\blacktriangle}$ +0.47 37/117
agentlessLineCoverage 33.07 32.95 $\color{red}{\blacktriangle}$ +0.12 32.95 $\color{red}{\blacktriangle}$ +0.12 36/117

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@juan-fernandez
juan-fernandez marked this pull request as ready for review July 17, 2026 10:15
@juan-fernandez
juan-fernandez requested review from a team as code owners July 17, 2026 10:15
@juan-fernandez
juan-fernandez requested review from AlexeyKuznetsov-DD and amarziali and removed request for a team July 17, 2026 10:15

@datadog-prod-us1-3 datadog-prod-us1-3 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: PASS

More details

Configuration parsing correctly handles all edge cases: empty entries are filtered by ConfigProvider.parseList before reaching the validation code, max-32-flags limit is enforced with fail-open warning, flags are immutably copied and properly serialized as JSON array in the upload event. Event map type changed from Map<String, String> to Map<String, Object> with Moshi adapter type updated accordingly. Test migration from Groovy to Java is complete with comprehensive edge-case coverage. No behavioral regressions detected.

Was this helpful? React 👍 or 👎

📊 Validated against 8 scenarios · Open Bits AI session

🤖 Datadog Autotest · Commit 34c0678 · What is Autotest? · Any feedback? Reach out in #autotest

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 17, 2026

Copy link
Copy Markdown

Test Environment - sonar-java

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent -20.48 7.81 $\color{green}{\blacktriangledown}$ -28.29 13.13 $\color{green}{\blacktriangledown}$ -33.61 37/126
agentless -21.64 11.65 $\color{green}{\blacktriangledown}$ -33.29 17.03 $\color{green}{\blacktriangledown}$ -38.67 37/125
agentlessCodeCoverage 37.79 73.34 $\color{green}{\blacktriangledown}$ -35.55 84.36 $\color{green}{\blacktriangledown}$ -46.57 37/125
agentlessLineCoverage 68.55 125.85 $\color{green}{\blacktriangledown}$ -57.30 136.34 $\color{green}{\blacktriangledown}$ -67.79 36/124

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@datadog-prod-us1-3

datadog-prod-us1-3 Bot commented Jul 17, 2026

Copy link
Copy Markdown

Bits couldn’t fix the CI

🟢 Investigated · 🔴 Fix failed · ⚪ Validate · ⚪ Ready

validate_supported_configurations_v2_local_file fails because the new DD_CODE_COVERAGE_FLAGS configuration is not registered in the central configuration registry. The local metadata entry is already present; registering the configuration externally is required before this validation can succeed.


View in Datadog | Reviewed commit 34c0678

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: ci visibility Continuous Integration Visibility tag: ai generated Largely based on code generated by an AI or LLM type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants