OCPBUGS-99052: recover from reverse proxy panic on browser disconnect

[logonoff]

Analysis / Root cause:

The API discovery endpoint (/api/api-discovery) uses httptest.NewRecorder to make in-process requests through the k8s reverse proxy. When a browser disconnects mid-request (e.g. page refresh or navigation), httputil.ReverseProxypanics withhttp.ErrAbortHandler` during response body copy. Unlike a real HTTP server, the recorder has no server to catch this panic, crashing the goroutine.

Reproducible by rapidly refreshing the browser directly on the api-discovery backend route on HTTP/1.1.

Solution description:

Add a defer/recover in k8sViaProxy that catches http.ErrAbortHandler specifically and converts it to an error return. Unexpected panics are re-raised. The first parameter type is widened from *proxy.Proxy to http.Handler to allow injecting a mock handler in tests.

Screenshots / screen recording:

N/A — backend-only fix, no UI changes.

Test setup:

No special setup required. Run go test ./pkg/server/ -run TestK8sViaProxy -v.

Test cases:

• TestK8sViaProxy_RecoversAbortPanic: Injects an http.HandlerFunc that panics with http.ErrAbortHandler (reproducing what httputil.ReverseProxy does on a broken connection) and verifies k8sViaProxy returns an error instead of panicking. Deterministically fails without the fix (panic: net/http: abort Handler).

Browser conformance:

• Chrome
• Firefox
• Safari (or Epiphany on Linux)

Additional info:

Stack trace from the crash:

goroutine 2879 [running]:
net/http/httputil.(*ReverseProxy).ServeHTTP(...)
  reverseproxy.go:613
github.com/openshift/console/pkg/proxy.(*Proxy).ServeHTTP(...)
  pkg/proxy/proxy.go:151
github.com/openshift/console/pkg/server.k8sViaProxy(...)
  pkg/server/api_discovery.go:114

Depends on CONSOLE-5426 (#16769).

Summary by CodeRabbit

• Bug Fixes
  • Improved handling of aborted Kubernetes API proxy requests by returning a clear error instead of terminating unexpectedly.
  • Preserved propagation of unexpected server errors for appropriate handling.

[openshift-merge-bot]

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

[openshift-ci-robot]

@logonoff: This pull request references Jira Issue OCPBUGS-99052, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (5.0.0) matches configured target version for branch (5.0.0)
• bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Analysis / Root cause:

The API discovery endpoint (/api/api-discovery) uses httptest.NewRecorder to make
in-process requests through the k8s reverse proxy. When a browser disconnects mid-request
(e.g. page refresh or navigation), httputil.ReverseProxy panics with
http.ErrAbortHandler during response body copy. Unlike a real HTTP server, the recorder
has no server to catch this panic, crashing the goroutine.

Reproducible by rapidly refreshing the browser on any page that triggers API discovery
(e.g. API Explorer) over HTTP/1.1.

Solution description:

Add a defer/recover in k8sViaProxy that catches http.ErrAbortHandler specifically
and converts it to an error return. Unexpected panics are re-raised. The first parameter
type is widened from *proxy.Proxy to http.Handler to allow injecting a mock handler
in tests.

Screenshots / screen recording:

N/A — backend-only fix, no UI changes.

Test setup:

No special setup required. Run go test ./pkg/server/ -run TestK8sViaProxy -v.

Test cases:

• TestK8sViaProxy_RecoversAbortPanic: Injects an http.HandlerFunc that panics with
  http.ErrAbortHandler (reproducing what httputil.ReverseProxy does on a broken
  connection) and verifies k8sViaProxy returns an error instead of panicking.
  Deterministically fails without the fix (panic: net/http: abort Handler).

Browser conformance:

• Chrome
• Firefox
• Safari (or Epiphany on Linux)

Additional info:

Stack trace from the crash:

goroutine 2879 [running]:
net/http/httputil.(*ReverseProxy).ServeHTTP(...)
 reverseproxy.go:613
github.com/openshift/console/pkg/proxy.(*Proxy).ServeHTTP(...)
 pkg/proxy/proxy.go:151
github.com/openshift/console/pkg/server.k8sViaProxy(...)
 pkg/server/api_discovery.go:114

Depends on CONSOLE-5426 (#16769).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[logonoff]

/verified by TestK8sViaProxy_RecoversAbortPanic
/label acknolwedge-critical-fixes-only

[openshift-ci]

@logonoff: The label(s) /label acknolwedge-critical-fixes-only cannot be applied. These labels are supported: acknowledge-critical-fixes-only, platform/aws, platform/azure, platform/baremetal, platform/google, platform/libvirt, platform/openstack, ga, tide/merge-method-merge, tide/merge-method-rebase, tide/merge-method-squash, px-approved, docs-approved, qe-approved, ux-approved, no-qe, rebase/manual, cluster-config-api-changed, run-integration-tests, verified, ready-for-human-review, approved, backport-risk-assessed, bugzilla/valid-bug, cherry-pick-approved, jira/skip-dependent-bug-check, jira/valid-bug, ok-to-test, plugin-api-approved, plugin-api-changed, stability-fix-approved, staff-eng-approved. Is this label configured under labels -> additional_labels or labels -> restricted_labels in plugin.yaml?

Details

In response to this:

/verified by TestK8sViaProxy_RecoversAbortPanic
/label acknolwedge-critical-fixes-only

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[openshift-ci-robot]

@logonoff: This PR has been marked as verified by TestK8sViaProxy_RecoversAbortPanic.

Details

In response to this:

/verified by TestK8sViaProxy_RecoversAbortPanic
/label acknolwedge-critical-fixes-only

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[logonoff]

/label acknowledge-critical-fixes-only

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: baf27de3-be95-4c08-a0d8-ff9a83dffcb4

📥 Commits

Reviewing files that changed from the base of the PR and between 16ccd09 and 38a8d7c.

📒 Files selected for processing (2)

• pkg/server/api_discovery.go
• pkg/server/api_discovery_test.go

🚧 Files skipped from review as they are similar to previous changes (2)

• pkg/server/api_discovery_test.go
• pkg/server/api_discovery.go

---

Walkthrough

k8sViaProxy now accepts any http.Handler, converts http.ErrAbortHandler panics into request errors, re-panics other values, and adds tests for both behaviors.

Changes

HTTP handler abort recovery

Layer / File(s)	Summary
Handler contract and panic recovery pkg/server/api_discovery.go	k8sViaProxy accepts an http.Handler, invokes ServeHTTP, converts http.ErrAbortHandler into an error, and preserves other panics.
Panic recovery validation pkg/server/api_discovery_test.go	Tests verify that abort panics return an error containing "request aborted" and unexpected panics propagate.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: jhadvig, leo6leo

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (14 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is specific, concise, and accurately summarizes the main fix to recover from reverse proxy panics on browser disconnects.
Description check	✅ Passed	The description covers root cause, solution, tests, and context; only the reviewers/assignees section is missing.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	The added subtest names are static, and no Ginkgo-style titles with dynamic values were added; the only variable t.Run uses fixed HTTP methods.
Test Structure And Quality	✅ Passed	The added test is a focused stdlib unit test, not Ginkgo; it uses no cluster resources or waits, has cleanup, and includes clear failure messages.
Microshift Test Compatibility	✅ Passed	No new Ginkgo e2e tests were added; the change only adds Go unit tests using httptest/http, with no MicroShift-specific APIs or unsupported features.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	Only standard Go unit tests were added in pkg/server; no new Ginkgo e2e test or multi-node/SNO assumption is present.
Topology-Aware Scheduling Compatibility	✅ Passed	Only backend API-discovery proxy logic and tests changed; no manifests, controllers, replicas, affinities, selectors, or PDBs were introduced.
Ote Binary Stdout Contract	✅ Passed	Touched files only add handler/test logic; no main/init/TestMain/RunSpecs or stdout-printing setup was added.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	No new Ginkgo e2e tests were added; the changed tests are standard httptest unit tests with no IPv4 literals or external connectivity.
No-Weak-Crypto	✅ Passed	Touched files only add panic recovery/test code; no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB or secret comparisons were introduced.
Container-Privileges	✅ Passed	PASS: Only backend Go files changed; no container/K8s manifest fields like privileged, hostPID/Network/IPC, SYS_ADMIN, or allowPrivilegeEscalation were added.
No-Sensitive-Data-In-Logs	✅ Passed	No sensitive data is logged; the added messages only mention generic API paths/statuses, and the test adds no logging.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[openshift-ci-robot]

@logonoff: This pull request references Jira Issue OCPBUGS-99052, which is valid.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (5.0.0) matches configured target version for branch (5.0.0)
• bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Details

In response to this:

Analysis / Root cause:

The API discovery endpoint (/api/api-discovery) uses httptest.NewRecorder to make
in-process requests through the k8s reverse proxy. When a browser disconnects mid-request
(e.g. page refresh or navigation), httputil.ReverseProxy panics with
http.ErrAbortHandler during response body copy. Unlike a real HTTP server, the recorder
has no server to catch this panic, crashing the goroutine.

Reproducible by rapidly refreshing the browser directly on the api-discovery backend route on HTTP/1.1.

Solution description:

Add a defer/recover in k8sViaProxy that catches http.ErrAbortHandler specifically
and converts it to an error return. Unexpected panics are re-raised. The first parameter
type is widened from *proxy.Proxy to http.Handler to allow injecting a mock handler
in tests.

Screenshots / screen recording:

N/A — backend-only fix, no UI changes.

Test setup:

No special setup required. Run go test ./pkg/server/ -run TestK8sViaProxy -v.

Test cases:

• TestK8sViaProxy_RecoversAbortPanic: Injects an http.HandlerFunc that panics with
  http.ErrAbortHandler (reproducing what httputil.ReverseProxy does on a broken
  connection) and verifies k8sViaProxy returns an error instead of panicking.
  Deterministically fails without the fix (panic: net/http: abort Handler).

Browser conformance:

• Chrome
• Firefox
• Safari (or Epiphany on Linux)

Additional info:

Stack trace from the crash:

goroutine 2879 [running]:
net/http/httputil.(*ReverseProxy).ServeHTTP(...)
 reverseproxy.go:613
github.com/openshift/console/pkg/proxy.(*Proxy).ServeHTTP(...)
 pkg/proxy/proxy.go:151
github.com/openshift/console/pkg/server.k8sViaProxy(...)
 pkg/server/api_discovery.go:114

Depends on CONSOLE-5426 (#16769).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[coderabbitai]

🧹 Nitpick comments (1)

pkg/server/api_discovery_test.go (1)

316-332: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Cover the unexpected-panic branch.

This test verifies http.ErrAbortHandler, but not the promised behavior that unrelated panics are re-raised. Add a second case that panics with another value and asserts that k8sViaProxy propagates it; otherwise a future recovery change could silently swallow real bugs while this test still passes.

This preserves the PR’s stated contract that unexpected panics are re-raised.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/server/api_discovery_test.go` around lines 316 - 332, Extend
TestK8sViaProxy_RecoversAbortPanic with a separate case using a panic value
other than http.ErrAbortHandler, and assert that k8sViaProxy re-panics with that
same value. Keep the existing aborted-request error assertions unchanged and
ensure the test fails if unexpected panics are swallowed.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@pkg/server/api_discovery_test.go`:
- Around line 316-332: Extend TestK8sViaProxy_RecoversAbortPanic with a separate
case using a panic value other than http.ErrAbortHandler, and assert that
k8sViaProxy re-panics with that same value. Keep the existing aborted-request
error assertions unchanged and ensure the test fails if unexpected panics are
swallowed.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 8f3f44f7-8813-4f33-bc30-ddf14488e5cf

📥 Commits

Reviewing files that changed from the base of the PR and between c11de39 and 16ccd09.

📒 Files selected for processing (2)

• pkg/server/api_discovery.go
• pkg/server/api_discovery_test.go

[logonoff]

/retest
/pipeline required

[openshift-merge-bot]

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-gcp-console
/test e2e-playwright

[logonoff]

/pipeline required
/verified by TestK8sViaProxy_RecoversAbortPanic

[openshift-ci-robot]

@logonoff: This PR has been marked as verified by TestK8sViaProxy_RecoversAbortPanic.

Details

In response to this:

/pipeline required
/verified by TestK8sViaProxy_RecoversAbortPanic

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@logonoff: This pull request references Jira Issue OCPBUGS-99052, which is valid.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (5.0.0) matches configured target version for branch (5.0.0)
• bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Details

In response to this:

Analysis / Root cause:

The API discovery endpoint (/api/api-discovery) uses httptest.NewRecorder to make in-process requests through the k8s reverse proxy. When a browser disconnects mid-request (e.g. page refresh or navigation), httputil.ReverseProxypanics withhttp.ErrAbortHandler` during response body copy. Unlike a real HTTP server, the recorder has no server to catch this panic, crashing the goroutine.

Reproducible by rapidly refreshing the browser directly on the api-discovery backend route on HTTP/1.1.

Solution description:

Add a defer/recover in k8sViaProxy that catches http.ErrAbortHandler specifically and converts it to an error return. Unexpected panics are re-raised. The first parameter type is widened from *proxy.Proxy to http.Handler to allow injecting a mock handler in tests.

Screenshots / screen recording:

N/A — backend-only fix, no UI changes.

Test setup:

No special setup required. Run go test ./pkg/server/ -run TestK8sViaProxy -v.

Test cases:

• TestK8sViaProxy_RecoversAbortPanic: Injects an http.HandlerFunc that panics with http.ErrAbortHandler (reproducing what httputil.ReverseProxy does on a broken connection) and verifies k8sViaProxy returns an error instead of panicking. Deterministically fails without the fix (panic: net/http: abort Handler).

Browser conformance:

• Chrome
• Firefox
• Safari (or Epiphany on Linux)

Additional info:

Stack trace from the crash:

goroutine 2879 [running]:
net/http/httputil.(*ReverseProxy).ServeHTTP(...)
 reverseproxy.go:613
github.com/openshift/console/pkg/proxy.(*Proxy).ServeHTTP(...)
 pkg/proxy/proxy.go:151
github.com/openshift/console/pkg/server.k8sViaProxy(...)
 pkg/server/api_discovery.go:114

Depends on CONSOLE-5426 (#16769).

Summary by CodeRabbit

• Bug Fixes
• Improved handling of aborted Kubernetes API proxy requests by returning a clear error instead of terminating unexpectedly.
• Preserved propagation of unexpected server errors for appropriate handling.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[TheRealJon]

/lgtm

[openshift-merge-bot]

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-gcp-console
/test e2e-playwright

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: logonoff, TheRealJon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• pkg/OWNERS [TheRealJon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment