Skip to content

Keep instrumenting classes decorated by another agent's synthetic types - #333

Open
Mishenevd wants to merge 1 commit into
mainfrom
fix/opentelemetry-agent-coexistence
Open

Keep instrumenting classes decorated by another agent's synthetic types#333
Mishenevd wants to merge 1 commit into
mainfrom
fix/opentelemetry-agent-coexistence

Conversation

@Mishenevd

Copy link
Copy Markdown
Collaborator

What

Zen silently stops instrumenting large parts of an app when it runs alongside the OpenTelemetry Java agent. In that setup SQL injection, SSRF, path traversal and command injection are no longer blocked, routes are not reported, and every request logs SpringAnnotationCollector: Received Spring Annotations, but no context set.

Reported by a customer running Zen v1.1.29 with the OTel agent v2.28.0 on Spring MVC.

Root cause

When the OTel agent is loaded first (-javaagent:opentelemetry-javaagent.jar -javaagent:zen.jar), it adds synthetic marker interfaces — its VirtualField accessors — onto core types such as java.sql.Connection/Statement/PreparedStatement, jakarta.servlet.Filter/Servlet and java.lang.Runnable.

Those synthetic types have no .class resource and live in OTel's own class loader. When Zen then transforms any class that implements one (PgStatement, RequestContextFilter, DispatcherServlet, …), Byte Buddy resolves the class hierarchy, fails to resolve the synthetic supertype, and throws TypePool$Resolution$NoSuchTypeException. The whole transform is aborted, so the class is left uninstrumented. This is only logged at trace level, so it is invisible in normal logs.

Losing the RequestContextFilter transform also removes the per-request context, which is why even sinks that are still instrumented (e.g. HttpURLConnection) stop blocking.

Fix

  • OtelCompatPoolStrategy: a type pool that delegates to the normal lazy pool but degrades a genuinely unresolvable type to an empty interface instead of throwing, so hierarchy resolution completes and our transform is applied.
  • Ignore io.opentelemetry.javaagent in the agent builder — those are the other agent's own classes, which we never need to instrument.

Testing

Reproduced and verified with the OpenTelemetry Java agent loaded before Zen, using the existing end-to-end suites (AIKIDO_BLOCK=1), comparing a Zen-enabled instance against a Zen-disabled one (safe request → 200, attack → 500 on enabled / 200 on disabled).

sample-apps/SpringBootPostgres + end2end/spring_boot_postgres.py (SQLi, command injection, SSRF, path traversal):

classes transformed resolution failures attacks blocked no context set
Zen only 36 0 all no
OTel → Zen (before) 29 205 none yes
OTel → Zen (after) 36 0 (12 on OTel-internal classes, now ignored) all no

Also re-run under OTel with the same before/after result on SpringBootHyperSQL, SpringMVCPostgresKotlin, SpringWebfluxSampleApp (WebFlux/reactor-netty) and JavalinPostgres — Spring MVC, WebFlux and a non-Spring framework.

Adds a runWithOpentel target (OTel loaded first) to SpringBootPostgres and a matching opentel.yml job so this cannot silently regress.

Notes

Comment thread agent/src/main/java/dev/aikido/agent/ByteBuddyInitializer.java Outdated
@Mishenevd
Mishenevd force-pushed the fix/opentelemetry-agent-coexistence branch 2 times, most recently from fa285cd to d87974b Compare August 10, 2026 13:36
Comment thread .github/workflows/opentel.yml
@codecov

codecov Bot commented Aug 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@Mishenevd
Mishenevd force-pushed the fix/opentelemetry-agent-coexistence branch from d87974b to 22822db Compare August 10, 2026 14:10
When the OpenTelemetry agent is loaded before Zen, it injects synthetic
supertypes (its VirtualField markers) onto core types such as java.sql.*,
jakarta.servlet.* and java.lang.Runnable. Those types have no .class resource,
so resolving the hierarchy of any class implementing one threw
NoSuchTypeException and the transform was skipped — silently turning off SQL
injection, request-context/route and executor instrumentation.

Add a lenient type pool that degrades an unresolvable type to an empty
interface so resolution completes, and skip OpenTelemetry's own classes.
@Mishenevd
Mishenevd force-pushed the fix/opentelemetry-agent-coexistence branch from 22822db to 9af214a Compare August 11, 2026 10:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant