OCIRepository fails to refresh public ghcr.io tags with 404 to github.com/-/v2/packages URL

[rubasace]

Describe the bug

OCIRepository resources targeting public charts on ghcr.io fail to refresh their tag digest after the first successful pull. The reconciler ends up making a request to a non-public GitHub URL (github.com/-/v2/packages/container/package/...) that returns 404, leaving the resource stuck in Ready=False indefinitely.

The image is publicly accessible (verified via curl with an anonymous token exchange), and the digest matches what is pinned in the resource.

Steps to reproduce

1. Apply the following on a cluster with source-controller v1.8.4 (Flux v2.8.7):

   apiVersion: source.toolkit.fluxcd.io/v1
   kind: OCIRepository
   metadata:
     name: dragonfly-operator
     namespace: flux-system
   spec:
     interval: 10m
     url: oci://ghcr.io/dragonflydb/dragonfly-operator/helm/dragonfly-operator
     timeout: 20m
     ref:
       tag: v1.5.0

2. Wait for the next reconcile cycle (≥ 10m).

3. Observe the controller logs and the resource status.

Actual behavior

OCIRepository reports Ready=False:

failed to determine artifact digest:
GET https://github.com/-/v2/packages/container/package/dragonflydb%2Fdragonfly-operator%2Fhelm%2Fdragonfly-operator%2Fmanifests%2Fv1.5.0:
unexpected status code 404 Not Found: Not Found

Note the URL is github.com/-/v2/packages/container/package/..., not the OCI distribution API endpoint ghcr.io/v2/.... This is an internal GitHub redirect target that is not a public OCI endpoint.

Reproducible with several public ghcr.io charts (e.g. dragonflydb/dragonfly-operator/helm/dragonfly-operator, weaveworks/charts/weave-gitops). Both produce the exact same error pattern.

Expected behavior

The reconciler should authenticate anonymously against ghcr.io (token endpoint ghcr.io/token with the right scope) and successfully retrieve the manifest, the same way docker pull / crane manifest / curl + Bearer do without credentials.

Evidence the image is reachable

Without auth (current source-controller behaviour):

curl -sI https://ghcr.io/v2/dragonflydb/dragonfly-operator/helm/dragonfly-operator/manifests/v1.5.0
# HTTP/2 401
# www-authenticate: Bearer realm="https://ghcr.io/token",
#                          service="ghcr.io",
#                          scope="repository:dragonflydb/dragonfly-operator/helm/dragonfly-operator:pull"

With anonymous token exchange:

TOKEN=$(curl -s 'https://ghcr.io/token?scope=repository:dragonflydb/dragonfly-operator/helm/dragonfly-operator:pull' \
  | jq -r .token)
curl -sI -H "Authorization: Bearer $TOKEN" \
        -H "Accept: application/vnd.oci.image.manifest.v1+json" \
        https://ghcr.io/v2/dragonflydb/dragonfly-operator/helm/dragonfly-operator/manifests/v1.5.0
# HTTP/2 200
# docker-content-digest: sha256:dd0656cdc1dc461e73336c88db5b664446621cdbcc65f0c44a184a4d34fe7c4c

So the registry path is correct and the manifest exists. The 401 → token → retry flow is what fails inside source-controller.

Workaround

Migrating the same chart to HelmRepository{type: oci} + HelmRelease.chart.spec works without any change in credentials or network setup, suggesting the code path used by HelmRepository{type: oci} handles the auth challenge correctly while OCIRepository does not.

Environment

• Flux version: v2.8.7
• source-controller: v1.8.4 (from gotk-components.yaml)
• Install method: bootstrap via flux install
• Kubernetes: v1.35 on Talos Linux v1.12.4
• Registry: ghcr.io, public repos, no credentials configured

Related closed issues

• [bug] Regression: OCIRepositories can no longer specify tag@digest  #1471 — same family (regression in tag@digest) — appears to be a different root cause but worth referencing.
• failed to determine artifact digest when creating ocirepository resource #1673 — similar error string but against Harbor + custom CA.

[rubasace]

After more testing on the same cluster (Flux v2.8.7 / source-controller v1.8.4),
the bug is narrower than I originally thought and the workarounds are clearer:

Matrix of OCIRepository.spec.ref configurations against ghcr.io

ref config	Endpoint hit by source-controller	Result
tag: v1.5.0@sha256:abc... (legacy combined syntax)	/v2/<repo>/manifests/v1.5.0	❌ 404 redirect to github.com/-/v2/packages/...
tag: v1.5.0 (tag only)	/v2/<repo>/manifests/v1.5.0	❌ same 404
semver: 1.5.0	/v2/<repo>/tags/list first, then manifest	✅ Ready=True
tag: v1.5.0 + digest: sha256:abc... (separate fields)	/v2/<repo>/manifests/sha256:abc... (digest wins precedence)	✅ Ready=True

Same registry, same chart, same anonymous auth, same controller version. Only
spec.ref differs.

The reveal is in .status.artifact.revision after reconcile:

• semver path: 0.4.0@sha256:abc... (tag prefix present → went through tag resolution)
• digest-pinned path: sha256:abc... (no tag prefix → resolved straight from digest endpoint)

Code-path confirmation

In internal/controller/ocirepository_controller.go:

• Tag-only path → getArtifactRef → remote.Head / remote.Get on repo.Tag(...) → hits /v2/<repo>/manifests/<tag> directly. This is the buggy path.
• Semver path → getTagBySemver → remote.List → hits /v2/<repo>/tags/list first.
• Digest path (separate ref.digest, gains precedence per docs) → resolves via repo.Digest(...) → hits /v2/<repo>/manifests/<digest>.

makeRemoteOptions is shared across all three, so transport/keychain are
identical. The bug is specifically in the manifests/<tag> endpoint flow
against ghcr.io.

Plausible mechanism

GitHub has documented since
Dec 2020
that ghcr.io redirects certain paths to the user-facing
github.com/<owner>/<repo>/pkgs/container/<name> page. The 404 URL in the
error (github.com/-/v2/packages/container/package/<repo>/manifests/<tag>)
looks exactly like the underlying ghcr→github redirect target that the OCI
client is following as if it were a valid registry response, instead of
treating the original 401 as an auth challenge and exchanging an anonymous
token. The tags/list and manifests/<digest> paths apparently don't
trigger that same redirect, so they work.

Workarounds

For anyone hitting this with public ghcr.io charts:

1. Best for production: use separate tag + digest fields. Keeps digest
   pinning (supply-chain protection against tag re-publish).

In this case, if like me, you are using Renovate, Renovate's flux manager bumps both fields atomically on new releases.

ref:
  tag: v1.5.0
  digest: sha256:dd0656cdc1dc461e73336c88db5b664446621cdbcc65f0c44a184a4d34fe7c4c

2. Simpler if you don't want digest pinning: use semver. In this case, Renovate's flux
   manager does not auto-bump ref.semver, so this is fine only if you
   pin to an exact version and accept manual bumps.

   ref:
     semver: 1.5.0

Suggested fix direction (cc @stefanprodan @hiddeco @matheuscscp)

If the tag resolution path is following an HTTP redirect that should have been
treated as a 401 challenge, I believe locking the OCI client to disallow cross-host
redirects on registry responses (or treating any redirect off ghcr.io as a
signal to fall back to the token endpoint) should make ref.tag behave the
same as the digest/semver paths against ghcr.io.

[matheuscscp]

@rubasace Which version of Renovate are you using? Renovate just shipped first-class support for OCIRepository.spec.ref.semver. I recommend switching:

renovatebot/renovate#41882

🎉 This PR is included in version 43.142.1 🎉

The release is available on:

• GitHub release
• 43.142.1

Your semantic-release bot 📦🚀

Now you can use .ref.semver along .ref.digest and Renovate will bump both.