Skip to content

fix(operator): skip no-op VirtualMCPServer Deployment updates - #6377

Open
RaviTharuma wants to merge 1 commit into
stacklok:mainfrom
RaviTharuma:fix/vmcp-skip-noop-deployment-update
Open

fix(operator): skip no-op VirtualMCPServer Deployment updates#6377
RaviTharuma wants to merge 1 commit into
stacklok:mainfrom
RaviTharuma:fix/vmcp-skip-noop-deployment-update

Conversation

@RaviTharuma

@RaviTharuma RaviTharuma commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Cross-links

Summary

VirtualMCPServer statusReportingInterval requeues (including the CRD 30s
default) called Update on the Deployment even when the pod template was
unchanged. That bumped metadata.generation and emitted DeploymentUpdated
with no new ReplicaSet — and only for vMCPs that had a unique
podTemplateSpec.

Root cause: podTemplateMetadataNeedsUpdate used maps.Equal on the full
label/annotation maps. User PodTemplateSpec merge leaves extra keys on the
live template, so every status tick looked like drift.

  • Compare pod-template metadata with MapIsSubset (expected ⊆ live)
  • Compare container env with equality.Semantic.DeepEqual (K8s defaulting)
  • Skip the Deployment write when the computed template, labels, merged
    annotations, and replicas are unchanged

Fixes #6340

Type of change

  • Bug fix
  • New feature
  • Refactoring (no behavior change)
  • Dependency update
  • Documentation
  • Other (describe):

Test plan

  • go test -ldflags=-extldflags=-Wl,-w ./cmd/thv-operator/controllers/ -run 'TestVirtualMCPServerEnsureDeployment_PodTemplateSpecSteadyState|TestVirtualMCPServerPodTemplateSpec'
  • CI unit / operator tests
  • After merge: a VirtualMCPServer with a unique podTemplateSpec should
    not emit DeploymentUpdated on every statusReportingInterval tick

API Compatibility

  • This PR does not break the v1beta1 API, OR the api-break-allowed label is applied and the migration guidance is described above.

No CRD schema change.

Does this introduce a user-facing change?

Yes. VirtualMCPServer Deployments no longer get a no-op Update (and a
generation bump) on every status-interval reconcile.

Special notes for reviewers

Matches the suggested fix on #6340: do not treat the status requeue itself as
drift. The subset check is what stops podTemplateSpec extras from looking
like a spec change; the skip-write is a second line of defense if
deploymentNeedsUpdate is still true for a non-template reason.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.03%. Comparing base (0fb54d4) to head (fb3753a).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #6377   +/-   ##
=======================================
  Coverage   78.02%   78.03%           
=======================================
  Files         767      767           
  Lines       74273    74273           
=======================================
+ Hits        57955    57959    +4     
+ Misses      16313    16309    -4     
  Partials        5        5           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@reyortiz3

Copy link
Copy Markdown
Collaborator

Nice fix for the root cause — the subset check + semantic env comparison directly address the drift false-positive from #6340, and the regression test follows the reconcile-twice/assert-ResourceVersion-unchanged pattern well.

One question on the "second line of defense" skip-write gate added in syncExistingVirtualMCPServerDeployment:

if equality.Semantic.DeepEqual(deployment.Spec.Template, newDeployment.Spec.Template) &&
    maps.Equal(deployment.Labels, newDeployment.Labels) &&
    maps.Equal(deployment.Annotations, mergedAnnotations) &&
    replicasUnchanged {
    return ctrl.Result{}, nil
}

newDeployment comes from deploymentForVirtualMCPServer, which never sets RestartPolicy, DNSPolicy, SchedulerName, etc. on the PodSpec. On a real cluster the live deployment.Spec.Template.Spec will have those API-server-defaulted, so this full-template DeepEqual will differ from newDeployment's essentially every time — meaning this gate won't actually fire in production once deploymentNeedsUpdate is true for any reason. The fake client used in TestVirtualMCPServerEnsureDeployment_PodTemplateSpecSteadyState doesn't apply that defaulting, so the test can pass without this path ever proving itself against real API-server behavior.

Since the actual fix already lives in podTemplateMetadataNeedsUpdate (subset check) and containerNeedsUpdate (semantic env compare) — both of which gate the call into this function in the first place — is this extra check pulling its weight? If it's meant as genuine defense-in-depth, could we validate it with an envtest (real API server, real defaulting) instead of/in addition to the fake-client test? Otherwise I'd lean toward dropping it to keep syncExistingVirtualMCPServerDeployment simpler, since as written it's effectively unreachable.

@reyortiz3

Copy link
Copy Markdown
Collaborator

Hi @RaviTharuma , can you please solve the conflicts?

@RaviTharuma

Copy link
Copy Markdown
Contributor Author

Hi @RaviTharuma , can you please solve the conflicts?

will do it, this weekend 👍

statusReportingInterval requeues were treated as drift when a user
PodTemplateSpec left extra labels on the live template. Compare
pod-template metadata as a subset and use semantic env equality so
generation stops bumping every interval.

The extra full-template DeepEqual skip-write gate is omitted: API-server
defaulting makes it unreachable, and the subset plus semantic-env checks
already prevent the write.

Fixes stacklok#6340

Co-authored-by: Ravi Tharuma <RaviTharuma@users.noreply.github.com>
@cursor
cursor Bot force-pushed the fix/vmcp-skip-noop-deployment-update branch from 9380995 to fb3753a Compare August 30, 2026 09:50
@RaviTharuma

Copy link
Copy Markdown
Contributor Author

@reyortiz3 Conflicts are resolved — the branch is MERGEABLE and CI is green.

On the extra skip-write gate: dropped it, as you suggested. The production path is deploymentNeedsUpdate → subset metadata checks + equality.Semantic.DeepEqual on env. A full-template DeepEqual against deploymentForVirtualMCPServer would keep missing API-server defaults, so it was not earning its keep as defense-in-depth. Happy to add an envtest later if you still want that coverage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(operator): VirtualMCPServer statusReportingInterval requeue Updates the Deployment every interval (no rollout)

3 participants