feat(telemetry): backport diagnostics metrics observability to 6.x

CHANGELOG.md

@@ -5,7 +5,29 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+## [6.53.0-beta.3]
+### Added
+- Diagnostics metrics observability, backported from the `master` (7.x) line: `DiagnosticsMetrics`
+  (`recordLatency`, `incrementCounter`, `setGauge`, `runWithBaseAttributes`), a split
+  traces/metrics/logs telemetry client (`@vtex/diagnostics-nodejs@0.1.8-io`,
+  `@vtex/diagnostics-semconv`), cluster resource attributes, and automatic Koa + host-metrics
+  instrumentation. Metrics are wired into the request pipeline itself, not just exposed as a
+  library API: HTTP handler latency/counters, request closed/aborted/total counters, outbound
+  HTTP client metrics, HTTP agent socket gauges, and the `@metric` GraphQL directive all emit
+  through `DiagnosticsMetrics` at the same points `master` does. Disabled by default; opt in
+  per app with `VTEX_DIAGNOSTICS_TELEMETRY_ENABLED=true`.
+### Fixed
+- No metrics/traces/logs from `6.x` apps ever reached ClickHouse, despite clean
+  "Telemetry enabled" initialization logs. Root cause (found via a temporary `debug: true`
+  diagnostic build): `yarn install` resolved `@grpc/grpc-js` into two separate copies —
+  `1.14.4` (satisfying `@vtex/diagnostics-nodejs`'s own `^1.13.4` requirement) and `1.13.3`
+  (satisfying the four `@opentelemetry/exporter-*-otlp-grpc` packages' `^1.7.1` requirement) —
+  a classic dual-package hazard: the `ChannelCredentials` object built by one copy failed an
+  `instanceof` check performed by the other (`TypeError: Channel credentials must be a
+  ChannelCredentials object`), so every export silently failed in the background. `master`
+  never hit this because its `yarn.lock` happened to collapse both ranges onto a single
+  `1.13.4` resolution. Fixed with a `resolutions` pin forcing `@grpc/grpc-js` to `1.13.4`
+  everywhere, matching `master`'s naturally-deduped resolution.
 
 ## [6.52.0]
 ### Added

__mocks__/@vtex/diagnostics-semconv.ts

@@ -0,0 +1,18 @@
+// Mock para @vtex/diagnostics-semconv
+const ATTR_VTEX_ACCOUNT_NAME = 'vtex.account.name'
+const ATTR_VTEX_IO_WORKSPACE_NAME = 'vtex_io.workspace.name'
+const ATTR_VTEX_IO_WORKSPACE_TYPE = 'vtex_io.workspace.type'
+const ATTR_VTEX_IO_APP_ID = 'vtex_io.app.id'
+const ATTR_VTEX_IO_APP_AUTHOR_TYPE = 'vtex_io.app.author-type'
+const ATTR_VTEX_IO_CLUSTER_ID = 'vtex_io.cluster.id'
+const ATTR_VTEX_IO_CLUSTER_ROLE = 'vtex_io.cluster.role'
+
+export {
+  ATTR_VTEX_ACCOUNT_NAME,
+  ATTR_VTEX_IO_WORKSPACE_NAME,
+  ATTR_VTEX_IO_WORKSPACE_TYPE,
+  ATTR_VTEX_IO_APP_ID,
+  ATTR_VTEX_IO_APP_AUTHOR_TYPE,
+  ATTR_VTEX_IO_CLUSTER_ID,
+  ATTR_VTEX_IO_CLUSTER_ROLE,
+}

jest.config.js

@@ -18,4 +18,13 @@ module.exports = {
   },
   testRegex: '(.*(test|spec)).tsx?$',
   testEnvironment: 'node',
+  // jest 25's bundled resolver predates package.json "exports" map support, so
+  // conditional-export-only subpaths (no legacy "main"-style file) fail to resolve
+  // even though Node itself resolves them fine at runtime. Map the ones pulled in
+  // transitively by @vtex/diagnostics-nodejs's OTLP gRPC exporters directly to their
+  // build output.
+  moduleNameMapper: {
+    '^@opentelemetry/otlp-exporter-base/node-http$':
+      '<rootDir>/node_modules/@opentelemetry/otlp-exporter-base/build/src/index-node-http.js',
+  },
 }

openspec/changes/add-observability-to-6x/tasks.md

@@ -51,28 +51,29 @@ Each numbered group below follows red → green → refactor: write the failing
 
 Deploying `6.53.0-beta.0` to the `iotest-ju2` test cluster with `DIAGNOSTICS_TELEMETRY_ENABLED=true` surfaced that no metrics were reaching ClickHouse. Root cause: groups 1–7 ported the `DiagnosticsMetrics` API and telemetry client, but never wired them into `6.x`'s actual request pipeline — `service/index.ts` never called `initializeTelemetry()`/set `global.diagnosticsMetrics`, and none of `master`'s five consumer call sites exist on `6.x` yet. This group closes that gap.
 
-- [ ] 8.1 **Red**: write/extend a test for `src/service/index.ts`'s `startApp()` asserting it calls `initializeTelemetry()` and sets `global.diagnosticsMetrics` to a `DiagnosticsMetrics` instance before serving requests; confirm it fails against the current implementation
-- [ ] 8.2 **Green**: update `startApp()` to call `await initializeTelemetry()` and set `global.diagnosticsMetrics = new DiagnosticsMetrics()`, matching `master`; declare the `global.diagnosticsMetrics` type augmentation
-- [ ] 8.3 **Red**: extend `src/service/worker/runtime/http/middlewares/timings.ts`'s test coverage with the "HTTP handler latency and counter" scenario (base attributes via `runWithBaseAttributes`, `recordLatency`, `incrementCounter('http_handler_requests_total', ...)`, graceful degradation when `global.diagnosticsMetrics` is unavailable); confirm it fails
-- [ ] 8.4 **Green**: port `master`'s `global.diagnosticsMetrics` emission logic into `timings.ts` to pass 8.3
-- [ ] 8.5 **Red**: extend `src/service/worker/runtime/http/middlewares/requestStats.ts`'s test coverage with the "Request lifecycle counters" scenario (closed/aborted/total); confirm it fails
-- [ ] 8.6 **Green**: port `master`'s `global.diagnosticsMetrics` emission logic into `requestStats.ts` to pass 8.5
-- [ ] 8.7 **Red**: extend `src/HttpClient/middlewares/metrics.ts`'s test coverage with the "Outbound HTTP client metrics" scenario; confirm it fails
-- [ ] 8.8 **Green**: port `master`'s `global.diagnosticsMetrics` emission logic into `HttpClient/middlewares/metrics.ts` to pass 8.7
-- [ ] 8.9 **Red**: extend `src/HttpClient/middlewares/request/HttpAgentSingleton.ts`'s test coverage with the "HTTP agent metrics" scenario; confirm it fails
-- [ ] 8.10 **Green**: port `master`'s `global.diagnosticsMetrics` emission logic into `HttpAgentSingleton.ts` to pass 8.9
-- [ ] 8.11 **Red**: extend `src/service/worker/runtime/graphql/schema/schemaDirectives/Metric.ts`'s test coverage with the "GraphQL `@metric` directive" scenario; confirm it fails
-- [ ] 8.12 **Green**: port `master`'s `global.diagnosticsMetrics` emission logic into `Metric.ts` to pass 8.11
-- [ ] 8.13 **Refactor**: confirm every emission point uses the same `if (global.diagnosticsMetrics) { ... } else { console.warn(...) }` guard shape as `master`, with no duplicated boilerplate beyond what `master` itself has
+- [x] 8.1 **Red**: added `src/service/index.test.ts` asserting `startApp()` calls `initializeTelemetry()` (before `startMaster`, via `invocationCallOrder`) and sets `global.diagnosticsMetrics` to a usable `DiagnosticsMetrics` instance; confirmed it fails on the missing `global.diagnosticsMetrics` type augmentation
+- [x] 8.2 **Green**: `startApp()` now `await initializeTelemetry()`s and sets `global.diagnosticsMetrics = new DiagnosticsMetrics()` before the master/worker branch, matching `master`; added the `NodeJS.Global` type augmentation. Note: the test asserts `initializeTelemetry` was called (not an exact count) because `DiagnosticsMetrics`'s own constructor also triggers it via `getMetricClient()` — the real singleton dedupes this, an exact-count assertion would just be testing a coincidence of the mock setup.
+- [x] 8.3 **Red**: ported `master`'s `src/service/worker/runtime/http/middlewares/timings.test.ts` verbatim (19 tests: base attributes, latency/counter recording, all status-code categories, graceful degradation); confirmed it fails against the pre-wiring `timings.ts`
+- [x] 8.4 **Green**: ported `master`'s `timings.ts` wiring (`runWithBaseAttributes`, `recordLatency`, `incrementCounter('http_handler_requests_total', ...)`) verbatim to pass 8.3
+- [x] 8.5 **Red**: ported `master`'s `requestStats.test.ts` verbatim; confirmed it fails against the pre-wiring `requestStats.ts`
+- [x] 8.6 **Green**: ported `master`'s `requestStats.ts` wiring (closed/aborted/total counters) verbatim to pass 8.5
+- [x] 8.7 **Red**: ported `master`'s `HttpClient/middlewares/metrics.test.ts` verbatim (13 tests); confirmed it fails against the pre-wiring `metrics.ts`
+- [x] 8.8 **Green**: ported `master`'s `metrics.ts` wiring, substituting `ACCOUNT_HEADER` for `master`'s `HeaderKeys.ACCOUNT` (the `HeaderKeys` refactor stays out of scope per design.md) to pass 8.7
+- [x] 8.9 **Red**: ported `master`'s `HttpAgentSingleton.test.ts` verbatim (covers the new `updateHttpAgentMetrics()` static method); confirmed it fails on the missing method
+- [x] 8.10 **Green**: added `HttpAgentSingleton.updateHttpAgentMetrics()` (gauges for sockets/free sockets/pending requests) to pass 8.9 — **plus one addition beyond the literal spec scenario list**: wired its only caller, `statusTrack.ts`'s `trackStatus()` (matching `master`), with a new test in `statusTrack.test.ts`; without this the method exists but nothing ever calls it periodically
+- [x] 8.11 **Red**: ported `master`'s `Metric.test.ts`, but added a new `describe` block that exercises the *real* `Metric` class directly (`Object.create(Metric.prototype)` + manual `.args`) — the ported test from `master` only reimplements the resolver logic inline and never imports the real class, so it would have passed trivially without any implementation change; confirmed the new block fails against the pre-wiring `Metric.ts`
+- [x] 8.12 **Green**: ported `master`'s `Metric.ts` wiring (`recordLatency`, `incrementCounter('graphql_field_requests_total', ...)`) to pass 8.11
+- [x] 8.13 **Refactor**: confirmed every emission point uses the same `if (global.diagnosticsMetrics) { ... } else { console.warn(...) }` guard shape as `master`, with no duplicated boilerplate beyond what `master` itself has
 
 ## 9. Full-suite regression and manual verification
 
 - [x] 9.1 Ran the complete `6.x` jest suite after groups 1–7: 88/88 tests pass across 8 suites; 1 pre-existing suite (`axiosTracing.test.ts`) fails on an unrelated TypeScript strictness error in `TestServer.ts` (`resolve()` called with no argument) — confirmed pre-existing via unchanged `yarn.lock` `typescript@4.9.5` resolution and a zero-diff on that file; not caused by this change
-- [ ] 9.2 Re-run the full jest suite after group 8 lands; confirm no regressions
-- [x] 9.3 Manually verify in a non-production workspace with `DIAGNOSTICS_TELEMETRY_ENABLED=true` — **done, and this is what surfaced the group-8 gap**: deployed `6.53.0-beta.0` to the `iotest-ju2` cluster; telemetry clients initialize (per the flag) but no per-request metrics reached ClickHouse, because nothing called the emission points. Re-verify after group 8 lands that `io_app_operation_duration_milliseconds` and the HTTP/GraphQL counters actually arrive.
-- [ ] 9.4 Manually verify with the flag unset in a live workspace: no telemetry initialization side effects (still `noop: true`) and app behavior unchanged from the pre-change baseline
+- [x] 9.2 Re-ran the full jest suite after group 8: 180/180 tests pass across 20 suites (up from 88/8 — the rebase onto `6.x`'s tip also picked up an unrelated Prometheus-aggregation backport's own new suites); same single pre-existing `axiosTracing.test.ts` failure, unchanged. `yarn build` compiles clean.
+- [x] 9.3 Manually verify in a non-production workspace with `DIAGNOSTICS_TELEMETRY_ENABLED=true` — **done, and this is what surfaced the group-8 gap**: deployed `6.53.0-beta.0` to the `iotest-ju2` cluster; telemetry clients initialize (per the flag) but no per-request metrics reached ClickHouse, because nothing called the emission points.
+- [ ] 9.4 Re-verify on `iotest-ju2` (or another test cluster) with a build that includes group 8: confirm `io_app_operation_duration_milliseconds`, `http_handler_requests_total`, `http_server_requests_*_total`, `http_client_requests_total`, `http_agent_*_current`, and `graphql_field_requests_total` all actually reach ClickHouse — not performed in this session, needs a live deploy
+- [ ] 9.5 Manually verify with the flag unset in a live workspace: no telemetry initialization side effects (still `noop: true`) and app behavior unchanged from the pre-change baseline — not performed in this session
 
 ## 10. Documentation and release
 
-- [x] 10.1 Added a `CHANGELOG.md` entry (currently under `[6.53.0-beta.0]`) on the `6.x` branch describing the diagnostics metrics capability and the `DIAGNOSTICS_TELEMETRY_ENABLED` flag — update this entry once group 8 lands to mention that metrics are now actually wired into the request pipeline, not just available as a library API
-- [ ] 10.2 Release a stable `6.x` version of `node-vtex-api` including this change, once group 8 is verified end-to-end on a test cluster
+- [x] 10.1 `CHANGELOG.md` entry exists under `[6.53.0-beta.0]` on the `6.x` branch describing the diagnostics metrics capability and the `DIAGNOSTICS_TELEMETRY_ENABLED` flag; updated to mention metrics are now wired into the request pipeline, not just available as a library API
+- [ ] 10.2 Release a stable `6.x` version of `node-vtex-api` including this change, once 9.4/9.5 are verified end-to-end on a test cluster

package.json

@@ -1,6 +1,6 @@
 {
   "name": "@vtex/api",
-  "version": "6.52.0",
+  "version": "6.53.0-beta.3",
   "description": "VTEX I/O API client",
   "main": "lib/index.js",
   "typings": "lib/index.d.ts",
@@ -47,9 +47,14 @@
   },
   "license": "MIT",
   "dependencies": {
+    "@opentelemetry/api": "^1.9.0",
+    "@opentelemetry/host-metrics": "0.35.5",
+    "@opentelemetry/instrumentation": "0.57.2",
+    "@opentelemetry/instrumentation-koa": "0.47.1",
     "@types/koa": "^2.11.0",
     "@types/koa-compose": "^3.2.3",
-    "@vtex/diagnostics-nodejs": "0.1.0-beta.10",
+    "@vtex/diagnostics-nodejs": "0.1.8-io",
+    "@vtex/diagnostics-semconv": "5.5.2",
     "@vtex/node-error-report": "^0.0.3",
     "@wry/equality": "^0.1.9",
     "agentkeepalive": "^4.0.2",
@@ -124,5 +129,8 @@
     "typemoq": "^2.1.0",
     "typescript": "^4.4.4",
     "typescript-json-schema": "^0.52.0"
+  },
+  "resolutions": {
+    "@grpc/grpc-js": "1.13.4"
   }
 }