Improving smoke test capabilities by PerfectSlayer · Pull Request #12051 · DataDog/dd-trace-java

PerfectSlayer · 2026-07-23T09:03:28Z

What Does This Do

This PR contains the follow up changes from all the other PRs of the stack.
In particular:

Add span link support to decoded spans and their assert rules
Remove test agent check customization (scope creep, not actionable in CI).

Motivation

Follow up changes from PR reviews listed in the additional notes.

Additional Notes

Related pull-requests:

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors
Once approved, use merge queue to merge the PR

Jira ticket: APMLP-1247

dd-octo-sts · 2026-07-28T09:14:47Z

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	14.74 s	14.67 s	[-0.5%; +1.3%] (no difference)
startup:insecure-bank:tracing:Agent	13.64 s	13.72 s	[-1.2%; +0.1%] (no difference)
startup:petclinic:appsec:Agent	17.09 s	16.95 s	[-0.0%; +1.7%] (no difference)
startup:petclinic:iast:Agent	16.93 s	17.03 s	[-1.4%; +0.2%] (no difference)
startup:petclinic:profiling:Agent	16.75 s	16.88 s	[-1.8%; +0.3%] (no difference)
startup:petclinic:sca:Agent	16.33 s	16.91 s	[-8.8%; +1.9%] (unstable)
startup:petclinic:tracing:Agent	16.18 s	16.02 s	[+0.1%; +1.9%] (maybe worse)

Commit: adcb95a8 · CI Pipeline · Benchmarking Platform UI

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

datadog-prod-us1-5 · 2026-07-28T09:34:29Z

View session in Datadog

Bits Code status: ✅ Done

Comment @DataDog to request changes

bric3 · 2026-08-28T14:42:40Z

Don't you think the final PR of the stack could sum up the whole work?
Would you rather have it limited to the last change only?

Having a summary of earlier work is OK, but the PR itself should precisely describe its change it tries to get in the repo not the previous ones. If the commit landing in git is too broad then it misleads. Gicen the title is used in git it should be tailored to the PR's changes.

In this case the commit will be "Improving smoke test capabilities" but it really is about span links harness tooling.

So yes I think the description should be focused on the changes. And possibly have a specific section that explain the complete set of work required to get there. Alternatively, I think it might be better to use an issue to track work.

PerfectSlayer · 2026-08-28T14:51:46Z

So yes I think the description should be focused on the changes.

At first, I thought about having another PR to stack for those changes, and keep this PR as the "preview full stack PR", still in draft. But as PR management / reviews become too painful, I did not want to go the clean way. I already baby sit PRs from 8am to 9pm so until we fix it, I'm not sure I will keep doing the "clean way". PRs are already way worst in the repo 😢

If the commit landing in git is too broad then it misleads

Commit log became significantly worse since merge queues… I don't use it anymore and not sure people still care about what is going in 😞 Anyway, I moved all the text behind HTLM comments so I can still read it but it won't be displayed.

bric3 · 2026-09-01T07:48:22Z

@PerfectSlayer Don't worry I get you 😢

ygree · 2026-09-01T18:24:06Z

@codex review

chatgpt-codex-connector · 2026-09-01T18:24:25Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-01T18:29:54.123005Z	`3b675f0`	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3b675f00e7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ygree

IDM mostly approves, except for a couple of places where I left notes

PerfectSlayer · 2026-09-05T12:17:21Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-09-05T12:17:26Z

View all feedbacks in Devflow UI.

2026-09-05 12:17:25 UTC ℹ️ Start processing command /merge

2026-09-05 12:17:30 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).

2026-09-05 12:38:26 UTC ❌ MergeQueue: The build pipeline contains failing jobs for this merge request

Build pipeline has failing jobs for 9b62f99:

Ubuntu_22_arm64.SIM: [test-app-java]

⚠️ Do NOT retry failed jobs directly (why?).

What to do next?

Investigate the failures and when ready, re-add your pull request to the queue!
If your PR checks are green, try to rebase/merge. It might be because the CI run is a bit old.
Any question, go check the FAQ.

Details

Since those jobs are not marked as being allowed to fail, the pipeline will most likely fail.
Therefore, and to allow other builds to be processed, this merge request has been rejected and the pipeline got canceled.

PerfectSlayer · 2026-09-05T12:39:40Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-09-05T12:39:45Z

View all feedbacks in Devflow UI.

2026-09-05 12:39:45 UTC ℹ️ Start processing command /merge

2026-09-05 12:39:50 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).

2026-09-05 12:56:45 UTC ❌ MergeQueue: The build pipeline contains failing jobs for this merge request

Build pipeline has failing jobs for b144ae4:

⚠️ Do NOT retry failed jobs directly (why?).

What to do next?

Investigate the failures and when ready, re-add your pull request to the queue!
If your PR checks are green, try to rebase/merge. It might be because the CI run is a bit old.
Any question, go check the FAQ.

Details

Since those jobs are not marked as being allowed to fail, the pipeline will most likely fail.
Therefore, and to allow other builds to be processed, this merge request has been rejected and the pipeline got canceled.

PerfectSlayer · 2026-09-05T13:01:13Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-09-05T13:01:17Z

View all feedbacks in Devflow UI.

2026-09-05 13:01:17 UTC ℹ️ Start processing command /merge

2026-09-05 13:01:22 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).

2026-09-05 13:49:49 UTC ❌ MergeQueue: The build pipeline contains failing jobs for this merge request

Build pipeline has failing jobs for 62eea7f:

Ubuntu_22_arm64.SID0: [test-app-java]

⚠️ Do NOT retry failed jobs directly (why?).

What to do next?

Investigate the failures and when ready, re-add your pull request to the queue!
If your PR checks are green, try to rebase/merge. It might be because the CI run is a bit old.
Any question, go check the FAQ.

Details

Since those jobs are not marked as being allowed to fail, the pipeline will most likely fail.
Therefore, and to allow other builds to be processed, this merge request has been rejected and the pipeline got canceled.

PerfectSlayer · 2026-09-05T18:02:03Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-09-05T18:02:07Z

View all feedbacks in Devflow UI.

2026-09-05 18:02:07 UTC ℹ️ Start processing command /merge

2026-09-05 18:02:13 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).

2026-09-05 18:50:11 UTC ℹ️ MergeQueue: This merge request was merged

PerfectSlayer added type: feature Enhancements and improvements comp: testing Testing tag: do not merge Do not merge changes labels Jul 23, 2026

PerfectSlayer force-pushed the bbujon/smoke-tests-tooling branch from 29804b8 to 817d668 Compare July 28, 2026 08:49

This comment has been minimized.

Sign in to view

PerfectSlayer force-pushed the bbujon/smoke-tests-tooling branch from 817d668 to c659547 Compare July 28, 2026 09:11

PerfectSlayer force-pushed the bbujon/smoke-tests-tooling branch from c659547 to 089af14 Compare July 29, 2026 06:09

PerfectSlayer mentioned this pull request Jul 30, 2026

Add APM test agent trace decoding #12089

Merged

PerfectSlayer force-pushed the bbujon/smoke-tests-tooling branch 6 times, most recently from f45d96b to 89f412f Compare August 7, 2026 08:13

PerfectSlayer force-pushed the bbujon/smoke-tests-tooling branch 2 times, most recently from e39c84d to adca11c Compare August 13, 2026 12:33

PerfectSlayer force-pushed the bbujon/smoke-tests-tooling branch 9 times, most recently from d28d501 to 18ce2e4 Compare August 27, 2026 09:13

PerfectSlayer changed the base branch from master to bbujon/smoke-tests-documentation August 27, 2026 09:14

PerfectSlayer removed the tag: do not merge Do not merge changes label Aug 27, 2026

PerfectSlayer marked this pull request as ready for review August 27, 2026 09:32

chatgpt-codex-connector Bot reviewed Sep 1, 2026

View reviewed changes

Comment thread dd-smoke-tests/src/main/java/datadog/smoketest/trace/SpanLinkMatcher.java Outdated

Comment thread ...st-agent-utils/decoder/src/main/java/datadog/trace/test/agent/decoder/json/raw/SpanJson.java Outdated

PerfectSlayer added 7 commits September 3, 2026 09:14

feat(testing): Help with static linter

11eba8c

feat(testing): Add span links support

a9f8b81

fix(testing): Drop custom test agent checks

c1211dd

fix(testing): Fix span id decode

38c23b0

chore(testing): Fix naming

fb83ea5

fix(testing): Fix span link any assert rule

bca8bb8

fix(testing): Fix JSON span link decoding

8559be0

PerfectSlayer force-pushed the bbujon/smoke-tests-tooling branch from acfa565 to 8559be0 Compare September 3, 2026 07:15

ygree reviewed Sep 3, 2026

View reviewed changes

Comment thread ...t/instrumentation-testing/src/main/java/datadog/trace/agent/test/assertions/SpanMatcher.java Outdated

ygree reviewed Sep 3, 2026

View reviewed changes

Comment thread dd-smoke-tests/src/main/java/datadog/smoketest/trace/SpanMatcher.java

ygree reviewed Sep 3, 2026

View reviewed changes

Comment thread utils/test-agent-utils/decoder/src/main/java/datadog/trace/test/agent/decoder/DecodedSpan.java

ygree approved these changes Sep 3, 2026

View reviewed changes

feat(testing): Improve control flow for static checker

adcb95a

gh-worker-dd-mergequeue-cf854d Bot merged commit 8ff7f46 into master Sep 5, 2026
600 checks passed

gh-worker-dd-mergequeue-cf854d Bot deleted the bbujon/smoke-tests-tooling branch September 5, 2026 18:50

github-actions Bot added this to the 1.66.0 milestone Sep 5, 2026

Uh oh!

Conversation

PerfectSlayer commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

This comment has been minimized.

dd-octo-sts Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟢 Java Benchmark SLOs — All performance SLOs passed

Uh oh!

datadog-prod-us1-5 Bot commented Jul 28, 2026 • edited by datadog-prod-us1-3 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bric3 commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PerfectSlayer commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bric3 commented Sep 1, 2026

Uh oh!

ygree commented Sep 1, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ygree left a comment

Choose a reason for hiding this comment

Uh oh!

PerfectSlayer commented Sep 5, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Sep 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What to do next?

Uh oh!

PerfectSlayer commented Sep 5, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Sep 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What to do next?

Uh oh!

PerfectSlayer commented Sep 5, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Sep 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What to do next?

Uh oh!

PerfectSlayer commented Sep 5, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Sep 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

PerfectSlayer commented Jul 23, 2026 •

edited

Loading

dd-octo-sts Bot commented Jul 28, 2026 •

edited

Loading

datadog-prod-us1-5 Bot commented Jul 28, 2026 •

edited by datadog-prod-us1-3 Bot

Loading

bric3 commented Aug 28, 2026 •

edited

Loading

PerfectSlayer commented Aug 28, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Sep 1, 2026 •

edited

Loading

gh-worker-devflow-routing-ef8351 Bot commented Sep 5, 2026 •

edited

Loading

gh-worker-devflow-routing-ef8351 Bot commented Sep 5, 2026 •

edited

Loading

gh-worker-devflow-routing-ef8351 Bot commented Sep 5, 2026 •

edited

Loading

gh-worker-devflow-routing-ef8351 Bot commented Sep 5, 2026 •

edited

Loading