feat(config): add DD_SERVERLESS_APM_ONLY traces-only mode#1293
feat(config): add DD_SERVERLESS_APM_ONLY traces-only mode#1293zarirhamza wants to merge 4 commits into
Conversation
This comment has been minimized.
This comment has been minimized.
Introduce a single master switch, DD_SERVERLESS_APM_ONLY, that puts the
extension in APM-only ("traces only") mode. When enabled, every billable
metrics and logs egress path is suppressed so the customer incurs no
infrastructure-monitoring or log-ingestion charges, while traces and APM
trace stats are unaffected.
Implemented on the datadog-agent-config Config<LambdaConfig> model:
- LambdaConfig / LambdaConfigSource gain a serverless_apm_only field, merged
from both env (DD_SERVERLESS_APM_ONLY) and YAML. Defaults to false.
- get_config() applies a post-merge override that forces
serverless_logs_enabled, enhanced_metrics, lambda_proc_enhanced_metrics and
the core OTLP metrics/logs toggles off, overriding any individually set
values so the guarantee holds regardless of other configuration.
Enforced defensively at the egress points as well:
- main: start_metrics_flushers returns no flushers, so custom DogStatsD /
enhanced / process metrics drained from the aggregator are discarded.
- logs flusher: flush() short-circuits to guarantee no log egress even if
something is queued.
Adds unit tests covering the default-off case and the forced-override case.
827c848 to
3f4006a
Compare
There was a problem hiding this comment.
Pull request overview
Adds a new master configuration switch, DD_SERVERLESS_APM_ONLY, to run Bottlecap in an APM-only (“traces only”) mode by suppressing logs and all metrics egress paths while leaving traces and trace stats unaffected.
Changes:
- Extends
LambdaConfig/LambdaConfigSourcewithserverless_apm_onlyand applies a post-merge override inget_config()to force-disable logs/metrics/OTLP logs+metrics toggles. - Prevents metrics egress by returning no metrics flushers when APM-only mode is enabled.
- Prevents logs egress by short-circuiting the logs flusher in APM-only mode; adds config-level tests for default/override behavior.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| bottlecap/src/config/mod.rs | Adds serverless_apm_only to config + applies override to force-disable logs/metrics/OTLP signals; includes new tests. |
| bottlecap/src/bin/bottlecap/main.rs | Disables metrics shipping by not starting metrics flushers in APM-only mode. |
| bottlecap/src/logs/flusher.rs | Disables log shipping by returning no requests from the logs flusher in APM-only mode. |
Address Copilot review feedback on the DD_SERVERLESS_APM_ONLY PR: - Drain and discard the logs aggregator in APM-only mode instead of returning early, keeping memory bounded and mirroring the metrics path (which drains its aggregator and discards the data). No log egress either way. - Add serverless_apm_only_from_yaml to guard the YAML key mapping, complementing the existing env-var override coverage.
duncanista
left a comment
There was a problem hiding this comment.
some integration testing would be good here, overall, not sure about the naming of the env var, is this going to be applied to every serverless product? does this type of env var exist outside of serverless?
…gration suite Address review feedback on the APM standalone (traces-only) PR: - Rename DD_SERVERLESS_APM_ONLY -> DD_APM_STANDALONE_ENABLED (field apm_standalone_enabled, fn apply_apm_standalone) to align with the "APM Standalone" product terminology. - Rescope the metrics-flusher comment so it only describes the metrics egress path (logs/OTLP suppression lives in apply_apm_standalone and the logs flusher guard), and drop the redundant debug line there — the single authoritative debug now lives in apply_apm_standalone. - Add an apm-standalone integration suite: a baseline function and a standalone (DD_APM_STANDALONE_ENABLED=true) function that both emit a trace, a custom metric, and logs. Asserts traces survive for both while metrics and logs are suppressed for the standalone function, even though DD_SERVERLESS_LOGS_ENABLED=true is inherited from the default env.
|
@duncanista thanks for the review — addressed your points in a1b1b89: Naming. Renamed Applied to every serverless product? That's the intent — the extension is serverless-only, so I dropped the Outside serverless? Not currently — this flag lives in the Lambda Integration testing. Added an
|
| // also dropped upstream (serverless_logs_enabled is forced off, so the | ||
| // processor never queues them), but this guard guarantees no log egress | ||
| // regardless of aggregator or redrive state. See DD_APM_STANDALONE_ENABLED. | ||
| if self.config.ext.apm_standalone_enabled { |
There was a problem hiding this comment.
isn't there like an "enabled" section where we'd just add this?
duncanista
left a comment
There was a problem hiding this comment.
LGTM – i'd cut down the amount of comments, they're too long, can be done with fewer words, overall, left another comment, would test it on actual resources/integration tests before merging!
Summary
Adds a single master switch —
DD_SERVERLESS_APM_ONLY— that puts the extension in APM-only ("traces only") mode. When enabled, every billable metrics and logs egress path is suppressed so the customer incurs no infrastructure-monitoring or log-ingestion charges, while traces and APM trace stats are unaffected.This closes the gap that made a true "traces only" configuration impossible before: there was no toggle to disable custom DogStatsD metrics, so even with logs and enhanced metrics turned off, custom metrics could still egress. A single, defensive flag now guarantees the behavior.
false— existing behavior is completely preserved.DD_SERVERLESS_APM_ONLY) and YAML.Behavior
Implementation (on the Config model)
LambdaConfig/LambdaConfigSourcegain aserverless_apm_onlyfield, merged from both env (DD_SERVERLESS_APM_ONLY) and YAML viamerge_fields!. Defaults tofalse.get_config()applies a post-merge override (apply_serverless_apm_only) that forcesserverless_logs_enabled,enhanced_metrics,lambda_proc_enhanced_metrics(on.ext) and the coreotlp_config_metrics_enabled/otlp_config_logs_enabledtoggles off — overriding any individually set values so the guarantee holds regardless of other configuration.Enforcement (defense in depth)
config/mod.rs::apply_serverless_apm_only) — forces the logs/metrics/OTLP toggles off after the shared env/yaml merge.main.rs::start_metrics_flushers) — returns no flushers, so anything drained from the DogStatsD aggregator (custom / enhanced / process) is discarded rather than sent.logs/flusher.rs::flush) — short-circuits to an empty request set, guaranteeing no log egress even if something is queued or redriven.Testing
cargo fmt --all -- --check— passes.cargo clippy --workspace --all-targets --features default— passes (no new warnings).cargo test— passes, including two new tests:serverless_apm_only_defaults_off— flag defaults off; other toggles unchanged.serverless_apm_only_forces_metrics_and_logs_off— flag overrides explicitly-enabled metrics/logs/OTLP toggles.Live validation on AWS Lambda
Built the layer (arm64) and deployed two fully-distinct Node.js 22 functions (Datadog Node layer + this extension), one with the flag off and one on, then invoked each and queried the Datadog API:
serverless-apm-full)serverless-apm-standalone)The extension's Lambda Telemetry API subscription also drops from
[Platform, Extension, Function]to[Platform]when the flag is on, confirming logs are cut off at the source.Risk / rollback
Low. Behavior is gated entirely behind a new flag that defaults to
false; when unset the code paths are unchanged. Rollback is leavingDD_SERVERLESS_APM_ONLYunset.Notes for reviewers