otel-demo: upgrade Docker Compose deployment to 3.0.0

[kylehounslow]

Description

Upgrade the OpenTelemetry Demo to 3.0.0 for the Docker Compose deployment.

• Bump IMAGE_VERSION/DEMO_VERSION to 3.0.0 and the flagd/postgres/valkey pins to the 3.0 set.
• Rewrite docker-compose.otel-demo.yml to the 3.0 core service set: drop accounting, fraud-detection, kafka, llm, product-reviews; add telemetry-docs; adopt the upstream 3.0 healthchecks, k6 load generator, and per-service service.criticality resource attributes. Preserve stack customizations (network/logging anchors, container names, memory limits, restart policy, telemetry pointed at the stack collector, image refs, port publishing, and the local-backends alerting-init service).
• Replace the Locust load-generator env with k6 (LOAD_GENERATOR_VUS, K6_TARGET_URL).
• Rename postgresql to astronomy-db; init.sql now provisions astronomy_db with astronomy_user/monitoring_user.
• Mount the new product-catalog/otel-config.yml; refresh demo.flagd.json to the 3.0 flag set.
• Update the vendored astronomy dashboards and otel-demo alert rules for the 3.0 service set: remove the product-reviews panels, drop fraud-detection/product-reviews from the service-telemetry variable list, repoint the product-catalog throughput panel to db_client_operation_duration_seconds_count, repoint the Avg amount charged panel from the payment-service log attribute request.amount.units.low (gone in 3.0) to the grpc.oteldemo.PaymentService/Charge span attribute demo.payment.amount, and drop product-reviews/accounting from the alert-rule matchers.
• Add the frontend-proxy backend env vars (CHATBOT_*, FIREPIT_*, OPAMP_SERVER_*) so the 3.0 envoy bootstrap does not crash-loop when those services are absent.

Validation: brought up the 3.0.0 demo via Docker Compose in an isolated project. All core demo services plus the stack collector reached healthy; traces, metrics, and logs flowed to the collector under k6 load. astronomy-db ran init.sql (astronomy_db, roles, seeded product catalog), flagd loaded the 3.0 flag set, and the alert rules parse with product-reviews/accounting dropped.

Scope: Docker Compose only. Kubernetes/Helm and AWS CDK/CLI are tracked in the linked issue. The frontend-proxy env-var requirement applies to those modes too and is documented there.

Issues Resolved

Part of #433 (this PR covers the Docker Compose scope only).

Testing

Ran stack locally and validated otel-demo metrics. See screenshots in comment below. #434 (comment)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[kylehounslow]

Still pending: Validate the Astronomy Shop dashboard renders fully in opensearch-dashboards. Will post screenshots once completed

[kylehounslow]

Screenshots

Astronomy Shop

Astronomy Shop - service telemetry

[kylehounslow]

@ps48 you're the best person for review here. Please take a look when you get the chance. Thanks!

[ps48]

Reviewed the Docker Compose otel-demo 3.0.0 upgrade for stack bring-up only (not the demo app code), against a local checkout of this branch.

docker compose config renders all 43 services with no warnings; Prometheus alert rules pass promtool, flagd/dashboards/init.sql are valid, the image pins are real, and there are no dangling references to the dropped services (kafka, product-reviews, llm, accounting, fraud-detection) or dependency cycles.

Minor / non-blocking notes

1. Alert-rules copy drift (info): only the Docker-Compose copy of otel-demo-alerts.yml was updated. The Helm chart copy (charts/observability-stack/files/rules-otel-demo/otel-demo-alerts.yml) still matches on the removed product-reviews / accounting / fraud-detection. This is consistent with the stated Docker-Compose-only scope (Helm tracked separately), but the two copies have now diverged — might be worth a one-line note so it isn't missed.
2. monitoring_user is unused in this scope (minor): init.sql provisions it (with pg_monitor) and .env defines POSTGRES_MONITORING_PASSWORD, but nothing in the Compose stack scrapes Postgres, so it's currently dead config here (mirrors upstream — presumably for a future Postgres receiver).
3. Nit: the trailing # defined in … comments on the POSTGRES_*_PASSWORD lines in .env are harmless under Compose v2 (verified stripped during interpolation), but would leak into the value if that file were ever sourced by a shell or consumed via env_file:. Moving the note to its own line above would be more robust.

---

Added on review — two docs made stale by this upgrade (in-scope)

A. Broken access point: the /loadgen/ "Load generator UI" (higher priority).
This branch carries otel-demo.mdx (stacked on #432), whose access-points table lists:

| http://localhost:8080/loadgen/ | Load generator UI |

The Locust → k6 migration in this PR makes that a dead link: k6 is headless, the load-generator service publishes no port, and frontend-proxy no longer has a /loadgen/ route (LOCUST_WEB_HOST/LOCUST_WEB_PORT are gone). Opening it yields a proxy error / 404. Since this PR is what invalidates the row, drop it (and any "Load generator UI" prose) here. The /feature row stays valid — flagd-ui is still present.

B. Stale memory breakdown in README.md (~L388–391).
The "OpenTelemetry Demo adds (~1.9 GB total)" list still enumerates services this PR removes:

• Kafka: ~500 MB — kafka is dropped in 3.0
• Java services (fraud-detection, ad, accounting): ~540 MB — only ad remains

So the ~1.9 GB / ~3.0 GB combined figures are overstated and the list is wrong. Update to the 3.0 core set (and note k6 replaced Locust).