Skip to content

opentelemetry-sdk: Add ability to refresh process sensitive Resource attributes#5280

Open
herin049 wants to merge 26 commits into
open-telemetry:mainfrom
herin049:feat/tracer-provider-update-resource
Open

opentelemetry-sdk: Add ability to refresh process sensitive Resource attributes#5280
herin049 wants to merge 26 commits into
open-telemetry:mainfrom
herin049:feat/tracer-provider-update-resource

Conversation

@herin049

@herin049 herin049 commented Jun 6, 2026

Copy link
Copy Markdown
Contributor

Description

This PR adds support for refreshing process sensitive SDK resources after os.fork(). It introduces provider level resource updates for traces, metrics, and logs, marks process resource detection as process sensitive and uses os.register_at_fork() to re-run any process sensitive resource detectors in child processes. This ensures attributes such as process.pid are updated for newly emitted spans, metrics, and logs after a fork.

Changes

  • Added process sensitive resource detector support for identifying detectors that should be re-run after fork.
  • Refresh SDK provider resources after os.fork() using os.register_at_fork().
  • Updated existing tracers/loggers and exported metric/log/span resources to use refreshed process attributes in child processes.
  • Added fork based subprocess tests for traces, metrics and logs.

Fixes #5279

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

uv run tox -e py314-test-opentelemetry-sdk

Does This PR Require a Contrib Repo Change?

  • Yes. - Link to PR:
  • No.

Checklist:

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added
  • Documentation has been updated

@herin049 herin049 requested a review from a team as a code owner June 6, 2026 20:35
@herin049 herin049 added Approve Public API check This label shows that the public symbols added or changed in a PR are strictly necessary sdk Affects the SDK package. labels Jun 6, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds fork-safety for process-sensitive SDK Resource attributes (e.g., process.pid) by re-running process-sensitive resource detectors in child processes and propagating refreshed resources into trace, metrics, and logs providers (including already-created tracers/loggers). It also introduces provider-level update_resource() APIs and adds subprocess-based fork tests.

Changes:

  • Add ResourceDetector.is_process_sensitive() and _get_process_sensitive_resource() to selectively re-run only process-sensitive detectors.
  • Add update_resource() to TracerProvider, MeterProvider, and LoggerProvider, and wire os.register_at_fork(after_in_child=...) to refresh resources in child processes.
  • Add fork-based subprocess tests plus direct update_resource() unit tests for traces, metrics, and logs.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
opentelemetry-sdk/src/opentelemetry/sdk/resources/init.py Adds process-sensitivity marker on detectors and helper to aggregate only process-sensitive resources.
opentelemetry-sdk/src/opentelemetry/sdk/trace/init.py Adds tracer/provider resource refresh on fork and public update_resource() that updates existing tracers.
opentelemetry-sdk/src/opentelemetry/sdk/metrics/_internal/init.py Adds meter provider resource refresh on fork and update_resource() support.
opentelemetry-sdk/src/opentelemetry/sdk/_logs/_internal/init.py Adds logger provider resource refresh on fork, update_resource(), and propagation to active loggers.
opentelemetry-sdk/tests/resources/test_resources.py Adds unit tests for process-sensitivity defaults and _get_process_sensitive_resource().
opentelemetry-sdk/tests/trace/test_trace.py Adds TracerProvider.update_resource() test and fork subprocess test.
opentelemetry-sdk/tests/trace/scripts/tracer_provider_resource_after_fork.py New subprocess script validating trace resource PID refresh after fork.
opentelemetry-sdk/tests/metrics/test_metrics.py Adds MeterProvider.update_resource() test and fork subprocess test.
opentelemetry-sdk/tests/metrics/scripts/meter_provider_resource_after_fork.py New subprocess script validating metric resource PID refresh after fork.
opentelemetry-sdk/tests/logs/test_logs.py Adds LoggerProvider.update_resource() test and fork subprocess test.
opentelemetry-sdk/tests/logs/scripts/logger_provider_resource_after_fork.py New subprocess script validating log resource PID refresh after fork.
.changelog/5280.added Changelog entry for the new fork/resource refresh capability.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread opentelemetry-sdk/src/opentelemetry/sdk/trace/__init__.py Outdated
Comment thread opentelemetry-sdk/src/opentelemetry/sdk/metrics/_internal/__init__.py Outdated
Comment thread opentelemetry-sdk/src/opentelemetry/sdk/_logs/_internal/__init__.py Outdated
Comment thread opentelemetry-sdk/src/opentelemetry/sdk/_logs/_internal/__init__.py
Comment thread opentelemetry-sdk/tests/trace/scripts/tracer_provider_resource_after_fork.py Outdated
@herin049 herin049 moved this to Ready for review in Python PR digest Jun 6, 2026
Comment thread opentelemetry-sdk/src/opentelemetry/sdk/_logs/_internal/__init__.py Outdated
Comment thread opentelemetry-sdk/src/opentelemetry/sdk/resources/__init__.py Outdated
Comment thread opentelemetry-sdk/src/opentelemetry/sdk/resources/__init__.py Outdated
Comment thread opentelemetry-sdk/src/opentelemetry/sdk/resources/__init__.py Outdated
@herin049 herin049 requested review from aabmass and emdneto July 1, 2026 04:02

@emdneto emdneto left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I tested with one of my workloads here. There are some function names that still contain the wording sensitive, but more of a nit.

@herin049

herin049 commented Jul 1, 2026

Copy link
Copy Markdown
Contributor Author

LGTM. I tested with one of my workloads here. There are some function names that still contain the wording sensitive, but more of a nit.

Thanks I'll remove all references of "sensitive" from the PR

@herin049

herin049 commented Jul 2, 2026

Copy link
Copy Markdown
Contributor Author

LGTM. I tested with one of my workloads here. There are some function names that still contain the wording sensitive, but more of a nit.

All usages of the phrase "sensitive" should be removed now.

@xrmx xrmx left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot spotted that this has an unexpected effect. That is that the post fork handler will override any manual set resource attributes. One could argue that I don't need a detector setting service.instance.id if I set it manually and so maybe documenting that behavior is enough? An alternative would be to track the source of the attributes and skip the update if they have been set manually but not sure if it's worth it.

@github-project-automation github-project-automation Bot moved this from Ready for review to Reviewed PRs that need fixes in Python PR digest Jul 8, 2026
@emdneto

emdneto commented Jul 10, 2026

Copy link
Copy Markdown
Member

Copilot spotted that this has an unexpected effect. That is that the post fork handler will override any manual set resource attributes. One could argue that I don't need a detector setting service.instance.id if I set it manually and so maybe documenting that behavior is enough? An alternative would be to track the source of the attributes and skip the update if they have been set manually but not sure if it's worth it.

That's true. I think we can document the behavior for now. @herin049 wdyt?

@herin049

Copy link
Copy Markdown
Contributor Author

Copilot spotted that this has an unexpected effect. That is that the post fork handler will override any manual set resource attributes. One could argue that I don't need a detector setting service.instance.id if I set it manually and so maybe documenting that behavior is enough? An alternative would be to track the source of the attributes and skip the update if they have been set manually but not sure if it's worth it.

I'll add a comment for now. I think it's more desirable to set a new UUID for service.instance.id post-fork even when the user has provided their own original service.instance.id. Otherwise you'd end up have multiple processes with the same service.instance.id which defeats the entire point of this resource attribute.

Another option is to add the ability to disable this detector. Then the user can create their own custom process-dependent detector which populates service.instance.id, which will be re-ran post-fork automatically.

@herin049

Copy link
Copy Markdown
Contributor Author

Copilot spotted that this has an unexpected effect. That is that the post fork handler will override any manual set resource attributes. One could argue that I don't need a detector setting service.instance.id if I set it manually and so maybe documenting that behavior is enough? An alternative would be to track the source of the attributes and skip the update if they have been set manually but not sure if it's worth it.

I've added the following comment under the service instance id resource detector:

Note: because this detector is process dependent, providers refresh it
automatically after a fork and merge the newly detected value on top of
the existing resource. This means that if a user explicitly sets
service.instance.id (e.g. via OTEL_RESOURCE_ATTRIBUTES or
Resource.create(attributes=...)), that value will be overwritten with a
newly generated UUID the next time the process forks. This differs from
Resource.create(), which does preserve user supplied overrides.

@herin049 herin049 requested a review from xrmx July 11, 2026 03:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Approve Public API check This label shows that the public symbols added or changed in a PR are strictly necessary sdk Affects the SDK package.

Projects

Status: Reviewed PRs that need fixes

Development

Successfully merging this pull request may close these issues.

Add ability to refresh process-sensitive Resource attributes

6 participants