Skip to content

fix(strands): tolerate the 1.60.0 invocation baggage detach when a stream is closed from another context - #278

Merged
sipercai merged 1 commit into
alibaba:mainfrom
RichardoMrMu:fix/strands-baggage-context-detach
Oct 22, 2026
Merged

sipercai merged 1 commit into
alibaba:mainfrom
RichardoMrMu:fix/strands-baggage-context-detach

Conversation

@RichardoMrMu

Copy link
Copy Markdown
Collaborator

What

loongsuite-instrumentation-strands now tolerates the framework-owned OpenTelemetry baggage detach
that Strands runs when a stream is closed from another context. This fixes the
loongsuite-instrumentation-strands-latest failures on Python 3.10-3.13.

Root cause

strands-agents 1.60.0 (published 2026-10-12T19:37Z) added an invocation level baggage context to
Agent.stream_async:

  • strands/agent/agent.py:1427-1432 - baggage_ctx = context_api.get_current(), set_baggage(...)
    per configured attribute, then baggage_token = context_api.attach(baggage_ctx)
  • strands/agent/agent.py:1479-1480 - finally: context_api.detach(baggage_token)

ContextVar.reset only accepts a token in the context that created it, so when the application
closes the async generator elsewhere the framework's finally runs in a copied context and
opentelemetry.context.detach logs Failed to detach context and raises
ValueError: <Token ...> was created in a different Context.

The plugin wraps Agent.stream_async and closes the wrapped stream, so closing it with
asyncio.create_task(stream.aclose()) (which copies the current context) reaches that finally from
another context. tests/test_instrumentor.py::test_generator_exit_from_early_break_cleans_state_and_next_sibling,
which asserts that stream cleanup leaves no detach failure behind, therefore started to fail with
1.60.0 only.

Observed while checking this branch (base 2026-10-13T03:26:35Z):

No existing issue tracks this regression: #54 is the original instrumentation feature request, and the
failure was first noticed while #275's CI ran (unrelated to that PR's changes).

Fix

New src/opentelemetry/instrumentation/strands/_compat.py:

  • BaggageContextCompat replaces strands.agent.agent.context_api with a proxy only when the
    installed framework accepts baggage_attributes (so strands-agents < 1.60.0 is left untouched),
    and restores the original binding on uninstrument().
  • _TolerantContextApi pairs every framework attach with a private ContextVar probe. detach
    resets the real token when the probe proves the token is still valid in the current context; when
    it is not, the impossible reset is skipped and nothing is logged. Tokens the proxy did not hand out
    are delegated unchanged.

Tolerating the exception after the fact is not enough: opentelemetry.context.detach logs
Failed to detach context before it re-raises, so the error record would still appear. The probe
decides before the call and uses only a private ContextVar, so the OTel logging path is never
entered.

_instrument installs the shim fail-open (a failure logs a warning and instrumentation continues),
_uninstrument restores the binding, and the README/CHANGELOG describe the behaviour.

Verification

Local runs at the same repository revision, only strands-agents differing (-p no:cacheprovider):

environment before this change with this change
strands-agents 1.60.0 1 failed, 31 passed (test_generator_exit_from_early_break_cleans_state_and_next_sibling, same ValueError as CI) 37 passed
strands-agents 1.59.0 green 35 passed, 2 skipped
strands-agents 1.50.2 (oldest) green 35 passed, 2 skipped

RED discriminator: with BaggageContextCompat.install disabled, the same test fails again on 1.60.0
(1 failed), so the shim is what makes it pass.

New tests cover the cross-context detach being tolerated while the value stays attached in the
attaching context, unrelated ValueError still propagating, frameworks without baggage_attributes
never being patched, the instrumentor patching and restoring context_api, and the framework's
invocation baggage attach still happening once per invocation without leaking afterwards.

python -m ruff check instrumentation-loongsuite/loongsuite-instrumentation-strands and
python -m ruff format --check are clean (ruff 0.14.1, as pinned in dev-requirements.txt).

Notes

  • The cross-context case replaces the framework's reset with one extra ContextVar.set binding on the
    framework's own binding; the plugin's own tokens keep their exact pairing.
  • Upstream could avoid this by detaching inside the context that attached the token, or by skipping
    the detach when the context changed; the shim keeps the instrumentation green until then.
  • Unrelated observation while testing: with the instrumentation installed, the framework's invocation
    baggage is current for invoke_agent span start, but the plugin's stream context hooks attach their
    own context, so nested plugin spans and model calls do not see that baggage. That is pre-existing
    behaviour and is unchanged here.

… close

strands-agents 1.60.0 attaches an OpenTelemetry baggage context in
Agent.stream_async and detaches it from the async generator's finally
block. That token is only valid in the context that started the stream,
so an application that closes the stream from another context - for
example asyncio.create_task(stream.aclose()), which copies the current
context - makes opentelemetry.context.detach log "Failed to detach
context" and raise ValueError.

The instrumentation now keeps the framework's module-level context_api
binding behind a proxy that pairs every attach with a private context
probe. When the probe shows that the token belongs to another context
the impossible reset is skipped, so closing a stream no longer records
that error; when the token is still usable it is reset exactly as
before, and attach/detach of tokens the proxy did not hand out are
delegated unchanged. Frameworks without the baggage attach
(strands-agents < 1.60.0) keep the untouched context API.

Verified locally with strands-agents 1.60.0 (37 passed, previously the
stream cleanup test failed with the reported ValueError), 1.59.0 and
1.50.2 (35 passed, 2 skipped each).
@CLAassistant

CLAassistant commented Oct 13, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@sipercai sipercai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Validated the exact head with before/after reproduction of cross-task stream closure, the real Strands/OpenAI SDK reasoning and tool loop (two user turns per session, concurrent streaming sessions), cancellation and clean sibling/context checks, and source-to-cloud trace readback. The detach regression is fixed without changing business output, span count or token totals. The original plugin suite also passes against Strands 1.62.0 (37 tests). The separate pre-existing reasoning/Agent-output field issues are tracked in #282. Fresh-cache tox -e precommit passes with a clean worktree.

@sipercai
sipercai merged commit a62d187 into alibaba:main Oct 22, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants