fix(gantry): add stopgap startup budget for large clusters

[Vaibhav Patel (vpatelsj)]

Summary

• Increase the built-in Gantry membership informer startup timeout from 30 seconds to 30 minutes.
• Add a /livez startup probe with a 1,900-second budget so liveness does not restart Gantry during initial informer synchronization.
• Add regression coverage and document the 5,003-node AKS validation.

Important

This is a stopgap validated at 5,003 nodes. It gives the current startup architecture more time, but does not reduce the per-pod full-cluster informer work.

Measured 5K result

Baseline deployment on 5,003 nodes had 47 Ready pods and 4,888 restarting pods. Observed failures exceeded the default 30-second membership synchronization deadline.

With the 30-minute process timeout and 31-minute-40-second startup-probe budget, the final revision reached 5,003 updated pods, 5,003 Ready pods and zero restarts on the final revision.

This run validated startup stability only. It did not validate image-distribution performance because the deployment retained the registry.example.com placeholder upstream.

Validation

• go test ./cmd/gantry ./deploy/gantry ./internal/gantry/config
• ./bin/golangci-lint run -c .golangci.yaml ./cmd/gantry/... ./deploy/gantry/... ./internal/gantry/config/...
• make gantry-manifests gantry-build

A repository-wide make gantry run was attempted locally but stopped during lint because the host lacks openssl/aes.h, required while typechecking the unrelated TPM simulator tests under cmd/agent.