feat(otel-collector): add statsdreceiver for StatsD/DogStatsD ingestion by arj22 · Pull Request #3100 · hyperdxio/hyperdx

arj22 · 2026-09-09T16:26:30Z

Summary

Adds the OpenTelemetry statsdreceiver to HyperDX's custom collector build, so users can ingest StatsD metrics directly without a separate bridge. Purely additive/opt-in — no default pipeline or behavior changes.

The docs call out one real limitation from upstream: this receiver keeps aggregation state per-instance with no cross-replica coordination, so it shouldn't be wired into HyperDX's multi-replica otel-collector Deployment — it needs a dedicated single-replica instance.

Test plan

Built the collector and confirmed statsdreceiver wasn't previously compiled in, then confirmed it's present after the change (otelcontribcol components).
Validated the documented config against the compiled binary and actually started the collector with it (reached "Everything is ready"), not just validate.

🤖 Generated with Claude Code

Compiles in `statsdreceiver` so StatsD and DogStatsD-formatted metrics can be ingested directly, without a separate StatsD-to-OTLP bridge. Documents that it doesn't support horizontally-scaled deployments (a real, upstream-documented limitation, not HyperDX-specific) and that it should not be wired into the OpAMP-managed otel-collector Deployment, which runs multiple replicas for OTLP ingest availability. Verified against the real compiled binary (pinned v0.155.0): confirmed `statsdreceiver` is not currently compiled in, added it, rebuilt, and confirmed the documented config both passes `otelcontribcol validate --config` and actually starts (`otelcontribcol --config ...`, reaching "Everything is ready. Begin running and processing data."), using the same multi-config merge the container's own entrypoint uses. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

vercel · 2026-09-09T16:26:37Z

@arj22 is attempting to deploy a commit to the HyperDX Team on Vercel.

A member of the Team first needs to authorize it.

changeset-bot · 2026-09-09T16:26:38Z

🦋 Changeset detected

Latest commit: bba51b6

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages

Name	Type
@hyperdx/otel-collector	Minor
@hyperdx/api	Minor
@hyperdx/app	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

greptile-apps · 2026-09-09T16:29:33Z

Greptile Summary

Adds opt-in StatsD and DogStatsD ingestion support to the custom OpenTelemetry Collector.

Compiles statsdreceiver into the collector distribution.
Documents standalone and OpAMP pipeline configuration, DogStatsD tag handling, and the single-replica constraint.
Exposes UDP port 8125 in image metadata while keeping Compose host publication disabled by default to avoid port conflicts.
The previous unconditional host-port binding has been fully removed.

Confidence Score: 5/5

The PR appears safe to merge; the outstanding default port-conflict issue was fully fixed by making UDP publication opt-in.

No new actionable failures or repository-rule violations remain. The resolved regression-coverage thread was explicitly accepted as follow-up work, the DogStatsD simple-tag documentation was corrected, and the previously unresolved unconditional port binding is now removed from both Compose configurations.

Important Files Changed

Filename	Overview
packages/otel-collector/builder-config.yaml	Adds the upstream StatsD receiver to the custom collector build.
packages/otel-collector/README.md	Documents configuration, port publication, DogStatsD tags, OpAMP behavior, and aggregation constraints.
docker-compose.yml	Provides an opt-in commented UDP mapping without reserving port 8125 in the default stack.
docker-compose.dev.yml	Provides the corresponding configurable, opt-in UDP mapping for development.
docker/otel-collector/Dockerfile	Advertises the StatsD UDP port in both standalone collector images without publishing it.
docker/hyperdx/Dockerfile	Advertises the StatsD UDP port in the all-in-one image.
.changeset/statsd-receiver.md	Records the additive receiver support as a minor collector release.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    A[StatsD or DogStatsD client] -->|UDP 8125, when published| B[statsd receiver]
    B --> C[Dedicated metrics/statsd pipeline]
    C --> D[memory_limiter]
    D --> E[batch]
    E --> F[ClickHouse exporter]

_{Reviews (5): Last reviewed commit: "fix(otel-collector): make statsd's compo..." | Re-trigger Greptile}

github-actions · 2026-09-09T16:30:26Z

Deep Review

✅ No critical issues found.

This is a purely additive, opt-in change: it compiles statsdreceiver into the collector build, adds a commented-out 8125/udp compose mapping plus EXPOSE 8125/udp metadata, and documents standalone + OpAMP wiring. Correctness review confirmed the load-bearing claims hold — the OpAMP metrics/statsd pipeline name is never touched by the remote config, the standalone [otlp/hyperdx, statsd] list drops no default receiver, and clickhouse/memory_limiter/batch are all available to the new pipeline. The changeset scope (@hyperdx/otel-collector only) is correct; the extra api/app bumps come from pre-existing changesets already on the branch. Nearly all prior-review feedback is resolved in the current checkout. The items below are recommended, not blocking.

🟡 P2 -- recommended

packages/otel-collector/README.md:364 -- The StatsD section recommends binding endpoint: 0.0.0.0:8125 but never notes that StatsD/UDP has no authentication mechanism, unlike the parallel Datadog section's ### Authentication subsection and otlp/hyperdx's token/OIDC coverage.
- Fix: Add one line stating the statsd receiver cannot be authenticated, so once the 8125/udp mapping is uncommented on a reachable interface anything that can reach it can inject arbitrary metrics; advise restricting exposure at the network/firewall layer.
- _{security, previous-comments}
packages/otel-collector/builder-config.yaml:54 -- No CI or smoke test asserts statsdreceiver stays compiled in or that the documented fragment ingests, so a future OTEL_COLLECTOR_VERSION bump could silently drop the receiver or break the example with nothing failing.
- Fix: Add a component-presence assertion (or a smoke-tests/otel-collector/ bats case emitting UDP to 8125 and asserting rows reach ClickHouse); the author's deferral to a follow-up matching the Add spanmetricsconnector for RED metrics from Datadog traces #3044 precedent is defensible, so treat as recommended rather than blocking.
- _{testing, correctness, previous-comments}

🔵 P3 nitpicks (2)

packages/otel-collector/README.md:382 -- The phrase "the receiver just starts and is wired to nothing" is inaccurate; the collector skips instantiation of receivers no pipeline references, so an unreferenced statsd receiver never starts or binds :8125.
- Fix: Mirror the accurate span-metrics wording two paragraphs up ("compiled in and wired to nothing") so anyone debugging isn't sent looking for a listener that never exists.
packages/otel-collector/README.md:348 -- The opt-in-port rationale ("a fixed host port mapping fails startup if something already owns 8125/udp") is duplicated verbatim across docker-compose.yml, docker-compose.dev.yml, and the README, so the three copies will drift when one is edited.
- Fix: Keep the full rationale in the README and reduce the two compose comments to a one-line pointer back to it.

Reviewers (6): correctness, security, previous-comments, project-standards, maintainability, testing.

Testing gaps: No automated assertion that statsdreceiver remains compiled in across OTEL_COLLECTOR_VERSION bumps; the documented standalone/OpAMP YAML fragments are verified only manually (otelcontribcol validate/run), not in CI.

github-actions · 2026-09-09T16:30:46Z

PR Review

5 finding(s): 🔴 0 critical · 🟠 1 major · 🔵 4 minor

5 posted as inline comment(s) on the changed lines.

_{Severity is the reviewer's own estimate and is used for ordering, not filtering.}

…laim - Removed the timer_histogram_mapping example claiming to route "timer/histogram-type" metrics to an exponential histogram. Verified against the pinned v0.155.0 source (factory.go's defaultTimerHistogramMapping): timer, histogram, AND distribution all default to a gauge observer, and the example only remapped `timing` - histogram/distribution metrics would still land as gauges, contradicting the comment. - Reframed the section around plain StatsD rather than DogStatsD specifically - the receiver handles any StatsD client; DogStatsD's tag extension is supported but not required, and centering the docs on one vendor's dialect was misleading and pulled in the now-removed, Datadog-distribution-specific histogram-mapping example. - Net simplification: fewer lines, no vendor-specific framing, no incorrect claim. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…e_simple_tags - Adding statsd to the default `metrics` pipeline via CUSTOM_OTELCOL_CONFIG_FILE silently no-ops in OpAMP supervisor mode (HyperDX's default deployment): opampController.ts overwrites that pipeline's receivers/exporters at runtime the same way it does for spanmetricsconnector (see the span metrics section above, hyperdxio#2351). Added a second example using a dedicated `metrics/statsd` pipeline name, which the remote config never touches, mirroring the existing OpAMP-mode span_metrics example. Verified against the real compiled binary by reconstructing opampController's generated base config as a static fixture and running `validate --config` against the merge. - Corrected the multi-replica warning: it was written as if OpAMP mode itself implies multiple replicas, but HyperDX's shipped default deployments run the OpAMP-managed collector as a single instance — the actual constraint is scaling *whichever* collector ingests statsd beyond one replica, independent of which mode it runs in. - Restored `enable_simple_tags: true` to both examples and qualified the intro claim about DogStatsD tag support: valued tags work with no extra config, but bare tags (`#mytag`) are silently dropped without this option. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

StatsD defaults to UDP, and neither the shipped compose files nor the collector/all-in-one Dockerfiles published or exposed 8125 - a user who added the documented statsd config verbatim on top of the default stack got no data and no error (UDP sends are fire-and-forget, so nothing logs the drop). Publish `8125:8125/udp` on the otel-collector service in docker-compose.yml and docker-compose.dev.yml (parameterized via HDX_DEV_OTEL_STATSD_PORT, matching the other dev port overrides), and add 8125/udp to the EXPOSE lines in docker/otel-collector/Dockerfile (both dev and prod stages) and docker/hyperdx/Dockerfile (all-in-one). statsd itself stays opt-in (nothing listens on the port until a pipeline config adds the receiver), so this is a no-op for anyone not using it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

An unconditional fixed host port mapping fails the whole otel-collector service's startup if anything else already owns 8125/udp (a local StatsD daemon or Datadog Agent commonly does) - since statsd itself is opt-in and off by default, this would have broken the default stack for anyone not using the feature, just for having it in their environment. Commented the mapping out by default in both compose files, matching the existing opt-in-port convention already used in the same files (db's 27017, ch-server's 8123/9000). Uncommenting it remains a one-line change, documented in the README section.

arj22 · 2026-09-10T22:42:00Z

@wrn14897 this is ready for review.

wrn14897

LGTM

github-actions Bot added the external label Sep 9, 2026

greptile-apps Bot reviewed Sep 9, 2026

View reviewed changes

Comment thread packages/otel-collector/builder-config.yaml

github-actions Bot reviewed Sep 9, 2026

View reviewed changes

Comment thread packages/otel-collector/README.md Outdated

Comment thread packages/otel-collector/README.md

Comment thread packages/otel-collector/builder-config.yaml

pulpdrew requested a review from wrn14897 September 9, 2026 16:41

greptile-apps Bot reviewed Sep 9, 2026

View reviewed changes

Comment thread packages/otel-collector/README.md Outdated

github-actions Bot reviewed Sep 9, 2026

View reviewed changes

Comment thread packages/otel-collector/README.md Outdated

Comment thread packages/otel-collector/README.md

Comment thread packages/otel-collector/README.md Outdated

Comment thread packages/otel-collector/README.md

greptile-apps Bot reviewed Sep 9, 2026

View reviewed changes

Comment thread packages/otel-collector/README.md

github-actions Bot reviewed Sep 9, 2026

View reviewed changes

Comment thread packages/otel-collector/README.md Outdated

Comment thread packages/otel-collector/README.md

Comment thread packages/otel-collector/README.md

arj22 and others added 2 commits September 9, 2026 22:15

Merge branch 'main' into feat/add-statsd-receiver

e1de31d

greptile-apps Bot reviewed Sep 10, 2026

View reviewed changes

Comment thread docker-compose.yml Outdated

github-actions Bot reviewed Sep 10, 2026

View reviewed changes

Comment thread docker-compose.dev.yml Outdated

Comment thread docker-compose.yml Outdated

Comment thread packages/otel-collector/README.md Outdated

Comment thread packages/otel-collector/README.md

github-actions Bot reviewed Sep 10, 2026

View reviewed changes

Comment thread packages/otel-collector/README.md

Comment thread packages/otel-collector/README.md

Comment thread packages/otel-collector/README.md

Comment thread docker-compose.dev.yml

Comment thread packages/otel-collector/builder-config.yaml

wrn14897 approved these changes Sep 10, 2026

View reviewed changes

Uh oh!

Conversation

arj22 commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

vercel Bot commented Sep 9, 2026

Uh oh!

changeset-bot Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

greptile-apps Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Flowchart

Uh oh!

Uh oh!

github-actions Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deep Review

🟡 P2 -- recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

arj22 commented Sep 10, 2026

Uh oh!

wrn14897 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

arj22 commented Sep 9, 2026 •

edited

Loading

changeset-bot Bot commented Sep 9, 2026 •

edited

Loading

greptile-apps Bot commented Sep 9, 2026 •

edited

Loading

github-actions Bot commented Sep 9, 2026 •

edited

Loading

github-actions Bot commented Sep 9, 2026 •

edited

Loading