test: add e2e test for OpenCensus to OpenTelemetry metrics migration by khrm · Pull Request #1735 · tektoncd/chains

khrm · 2026-06-29T09:54:42Z

Adds TestOTelMetrics, a consolidated e2e test for the OC→OTel metrics migration in Chains (PR #1550). The test creates a TaskRun and a PipelineRun, waits for Chains to sign them, then scrapes the controller /metrics endpoint via port-forward to assert:

All signing counters increment (sign_created, payload_stored, marked_signed) for both TaskRun and PipelineRun
Knative workqueue metrics use the new kn_workqueue_* prefix
Standard go_* runtime metrics are present
Old OpenCensus metric names (tekton_chains_*) are absent

Uses expfmt/dto.MetricFamily for proper Prometheus text parsing and a baseline scrape before creating resources to compute deltas, avoiding false positives from earlier test runs.

Relates to #1550

Changes

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

Has Docs included if any changes are user facing
Has Tests included if any functionality added or changed
Follows the commit message standard
Meets the Tekton contributor standards (including
functionality, content, code)
Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings)
Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

NONE

infernus01 · 2026-07-01T09:56:39Z

+	removedMetrics := []string{
+		"tekton_chains_taskrun_signed_total",
+		"tekton_chains_pipelinerun_signed_total",
+	}


We can probably remove these, these metrics never existed I believe. This will pass in all cases.

@khrm as per earlier opencensus based implementation Knative's sharedmain uses the component name as the OC metrics domain/namespace, which gets prepended as a Prometheus prefix. So the old OC Prometheus names were already watcher_taskrun_sign_created_total, watcher_taskrun_payload_stored_total, etc. identical to the new OTel names. If we prefer to check for removed metric names recommend that to be on the knative metric names which were like watcher_workqueue_, etc., instead of kn_ now. Otherwise please ignore this block to verify removed metrics altogether

jkhelil · 2026-07-02T08:51:12Z

/retest

jkhelil · 2026-07-02T08:57:41Z

/ok-to-test

jkhelil · 2026-07-03T05:03:19Z

/retest

jkhelil · 2026-07-03T09:08:02Z

@khrm e2e are failing, can you have a look please

anithapriyanatarajan · 2026-07-06T06:10:35Z

@khrm PR #1738 has been merged to main and fixes the E2E CI failures. Please rebase your branch onto main so the E2E tests can pass.

anithapriyanatarajan · 2026-07-15T06:34:37Z

Closed and reopened to rerun the CI

codecov-commenter · 2026-07-15T06:37:52Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@1c7dcb5). Learn more about missing BASE report.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1735   +/-   ##
=======================================
  Coverage        ?   61.95%           
=======================================
  Files           ?       64           
  Lines           ?     4071           
  Branches        ?        0           
=======================================
  Hits            ?     2522           
  Misses          ?     1268           
  Partials        ?      281

Flag	Coverage Δ
unit-tests	`61.95% <ø> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

anithapriyanatarajan · 2026-07-15T07:50:44Z

+	removedMetrics := []string{
+		"tekton_chains_taskrun_signed_total",
+		"tekton_chains_pipelinerun_signed_total",
+	}


@khrm as per earlier opencensus based implementation Knative's sharedmain uses the component name as the OC metrics domain/namespace, which gets prepended as a Prometheus prefix. So the old OC Prometheus names were already watcher_taskrun_sign_created_total, watcher_taskrun_payload_stored_total, etc. identical to the new OTel names. If we prefer to check for removed metric names recommend that to be on the knative metric names which were like watcher_workqueue_, etc., instead of kn_ now. Otherwise please ignore this block to verify removed metrics altogether

anithapriyanatarajan · 2026-07-28T14:31:55Z

@khrm - could you get this minor comment - #1735 (comment) resolved. we could merge this. Thank you

Adds TestOTelMetrics, a consolidated e2e test for the OpenCensus-to- OpenTelemetry metrics migration in Chains (PR tektoncd#1550). The test creates a TaskRun and a PipelineRun, waits for Chains to sign them, then scrapes the controller /metrics endpoint via port-forward to assert: - All signing counters increment for both TaskRun and PipelineRun (sign_created, payload_stored, marked_signed) - Knative workqueue metrics use the new kn_workqueue_* prefix - Standard go_* runtime metrics are present - The old Knative workqueue prefix (watcher_workqueue_*) is absent after the rename to kn_workqueue_* Prometheus text output is parsed with expfmt/dto.MetricFamily, and a baseline scrape taken before creating resources is used to compute counter deltas, avoiding false positives from earlier test runs. The Chains signing counters keep identical Prometheus names across the migration (Knative sharedmain prepends the component name as the OpenCensus domain), so only Knative's own infrastructure metrics are renamed; the test verifies that rename rather than checking for names that never existed. Also replaces the kodata LICENSE symlink and removes the HEAD/refs symlinks so ko v0.19.0's strict enforcement that kodata symlinks must not resolve outside the kodata root does not fail the build (ko-build/ko#1619). Relates to tektoncd#1550 Co-authored-by: Anitha Natarajan <anataraj@redhat.com> Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Khurram Baig <kbaig@redhat.com> Signed-off-by: Anitha Natarajan <anataraj@redhat.com>

anithapriyanatarajan · 2026-08-13T16:21:32Z

@jkhelil @infernus01 - The comments are addressed. Could you take a look when you get sometime. Thank you

infernus01

/lgtm

tekton-robot · 2026-08-20T12:49:31Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: anithapriyanatarajan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [anithapriyanatarajan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

tekton-robot requested review from ab-ghosh and vdemeester June 29, 2026 09:54

tekton-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jun 29, 2026

khrm force-pushed the e2e-otel-metrics-test branch 6 times, most recently from 292afe2 to e521dd8 Compare June 30, 2026 15:58

infernus01 reviewed Jul 1, 2026

View reviewed changes

anithapriyanatarajan closed this Jul 15, 2026

anithapriyanatarajan reopened this Jul 15, 2026

anithapriyanatarajan requested changes Jul 15, 2026

View reviewed changes

anithapriyanatarajan force-pushed the e2e-otel-metrics-test branch from 01eab3c to 8b62105 Compare August 11, 2026 07:39

anithapriyanatarajan requested a review from infernus01 August 11, 2026 08:47

infernus01 reviewed Aug 14, 2026

View reviewed changes

tekton-robot assigned infernus01 Aug 14, 2026

tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Aug 14, 2026

anithapriyanatarajan approved these changes Aug 20, 2026

View reviewed changes

tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 20, 2026

tekton-robot merged commit e44b60f into tektoncd:main Aug 20, 2026
22 of 23 checks passed

Uh oh!

Conversation

khrm commented Jun 29, 2026

Changes

Submitter Checklist

Release Notes

Uh oh!

infernus01 Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

anithapriyanatarajan Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

jkhelil commented Jul 2, 2026

Uh oh!

jkhelil commented Jul 2, 2026

Uh oh!

jkhelil commented Jul 3, 2026

Uh oh!

jkhelil commented Jul 3, 2026

Uh oh!

anithapriyanatarajan commented Jul 6, 2026

Uh oh!

anithapriyanatarajan commented Jul 15, 2026

Uh oh!

codecov-commenter commented Jul 15, 2026 • edited by codecov Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

anithapriyanatarajan Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

anithapriyanatarajan commented Jul 28, 2026

Uh oh!

anithapriyanatarajan commented Aug 13, 2026

Uh oh!

infernus01 left a comment

Choose a reason for hiding this comment

Uh oh!

tekton-robot commented Aug 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

codecov-commenter commented Jul 15, 2026 •

edited by codecov Bot

Loading