ate-api-server's --egress-gateway-spiffe-id is repeatable, and ate-setup sets it from ATE_EGRESS_GATEWAY_EXTRA_SPIFFE_IDS by thompsonmax · Pull Request #3360 · agent-substrate/substrate

Max Thompson (thompsonmax) · 2026-10-27T23:52:44Z

Why is this needed?

Credential plugins may need an actor JWT to exchange for other credentials, e.g. through an RFC 8693 exchange. Today only egress_gateway can call MintActorJwt; ate-api-server grants that role to one SPIFFE ID. It's not possible to grant permissions to generate Actor JWTs to any other identity.

This makes --egress-gateway-spiffe-id repeatable and grants egress_gateway to every ID passed. An operator who deploys a plugin lists its SPIFFE ID in ate-setup's new ATE_EGRESS_GATEWAY_EXTRA_SPIFFE_IDS setting (--egress-gateway-extra-spiffe-ids). ate-setup then passes atenet-egress's ID first, since setting the flag replaces the default, followed by the listed IDs. The default is still atenet-egress alone, and an empty ID still fails startup.

There is no API or manifest change. A listed ID gets the whole egress_gateway role: besides minting actor JWTs, it can get and list actors, read their egress policies and mint actor certificates.

Part of #1661.

AI use: AI-assisted draft using Claude.

Tests pass
Docs updated

Shruti Nair (SHRUTI6991) · 2026-10-28T02:56:20Z

 	// over mTLS.
 	controllerSPIFFEID       = pflag.String("controller-spiffe-id", installdefaults.ControllerSPIFFEID(installdefaults.NamespaceFromPodEnv()), "mTLS SPIFFE ID of ate-controller. Holds the controller authz role, which manages workers.")
-	egressGatewaySPIFFEID    = pflag.String("egress-gateway-spiffe-id", installdefaults.EgressSPIFFEID(installdefaults.NamespaceFromPodEnv()), "mTLS SPIFFE ID of atenet-egress. Holds the egress_gateway authz role, which reads and lists actors and reads their egress policies.")
+	egressGatewaySPIFFEIDs   = pflag.StringArray("egress-gateway-spiffe-id", []string{installdefaults.EgressSPIFFEID(installdefaults.NamespaceFromPodEnv())}, "mTLS SPIFFE IDs of atenet-egress and any other client that holds the egress_gateway authz role, which reads and lists actors and reads their egress policies. Repeat the flag for each ID.")


I am little confused. Why are we not creating a separate credential plugin role to mint actor JWT?

Is this more work? If so, can you add a TODO to move this after GA? It seems weird to have any other systems be integrated in egress gateway role.

An earlier draft did exactly that. But I think we want to minimize the complexity of changes we're checking in this close to the v1.0 cut.

Eventually we will support RFC 8693 credential injection within the egress gateway rather than require a credential provider plugin (#1661), so I don't think it's worth the effort of creating/maintaining a separate role long-term.

Add a credential_minter system role that may mint actor JWTs and nothing else. ate-api-server grants it to --credential-minter-spiffe-id, which defaults to the credential-minter ServiceAccount in the install's namespace. Access policies cannot grant it, and an empty flag grants it to no one. A credential plugin that exchanges actor JWTs for cloud access tokens can then run as its own Deployment instead of inside the atenet-egress pod. Anyone who can run a pod as an arbitrary ServiceAccount in the install's namespace can already run one as atenet-egress, so the default grants no new authority. Signed-off-by: Max Thompson <maxthomp@google.com>

Replace --credential-minter-spiffe-id with --credential-minter-spiffe-ids so more than one credential plugin can hold the credential_minter role. The default is still the credential-minter ServiceAccount in the install's namespace, and an empty list grants the role to no one. Signed-off-by: Max Thompson <maxthomp@google.com>

--egress-gateway-spiffe-id now takes a list, comma-separated or repeated, and grants egress_gateway to every ID in it. A credential plugin that exchanges actor JWTs for cloud access tokens can then run as its own Deployment instead of inside the atenet-egress pod. The default is still atenet-egress alone, and an empty list fails startup. Signed-off-by: Max Thompson <maxthomp@google.com>

Make --egress-gateway-spiffe-id a StringArray so each value is one SPIFFE ID. StringSlice also split values on commas. Signed-off-by: Max Thompson <maxthomp@google.com>

Move the arg rewrite out of setAPIServerAuthzArgs into setAPIServerFlag so other settings can render ate-api-server flags. Signed-off-by: Max Thompson <maxthomp@google.com>

ATE_EGRESS_GATEWAY_EXTRA_SPIFFE_IDS lists SPIFFE IDs that hold ate-api-server's egress_gateway role next to atenet-egress, such as a credential provider that mints actor JWTs. ate-setup passes atenet-egress first because --egress-gateway-spiffe-id replaces its default when set. Signed-off-by: Max Thompson <maxthomp@google.com>

Tim Hockin (thockin)

Approving to make RC1, but I feel like the flag help and docs could be clearer, and flag validation failures should be more systematic

Tim Hockin (thockin) · 2026-10-29T00:48:44Z

 func parseSystemPrincipals(system SystemPrincipals) (map[string][]string, error) {
-	bindings := []struct {
+	if len(system.EgressGatewaySPIFFEIDs) == 0 {
+		return nil, fmt.Errorf("no %s SPIFFE ID", RoleEgressGateway)


Shouldn't we be doing flag validation WAY earlier than this?

Tim Hockin (thockin) · 2026-10-29T00:50:14Z

 | `--cordon-system-infra-components` | Keep the system infra components, such as ate-api-server, atenet-router, atenet-egress and postgres, off the worker nodes. Assumes a small shared pool labeled and tainted `ate.dev/workloadType=ate-system-infra:NoSchedule`, across which each workload's replicas are spread, and a one-node pool labeled and tainted `ate.dev/workloadType=ate-postgres:NoSchedule` for postgres alone. `ATE_INSTALL_CORDON_SYSTEM_INFRA_COMPONENTS=true` when the flag is absent |
 | `--fake-data-plane` | Install the fake data plane for control-plane benchmarks: fake-atelet takes atelet's place in its DaemonSet, taking its objectstore-plugin's storage settings, ate-controller is scaled to zero, and fake-workersync honors every WorkerPool with fake Workers on the fake-atelet nodes. Reaches `deploy ate-system`, `deploy atelet`, and `deploy ate-controller`; only `deploy ate-system` switches a cluster between the real and fake data planes, and the other two refuse a data plane other than the installed one. `deploy ate-system --fake-data-plane` refuses a cluster whose WorkerPools have worker pods. fake-atelet runs with its defaults; a benchmark tunes it through the optional `fake-atelet-tuning` ConfigMap, whose `FAKE_ATELET_*` variables set its flags, and which no install creates or resets. Actors do not exist; never use it on a cluster that serves real actors. `ATE_INSTALL_FAKE_DATA_PLANE=true` when the flag is absent |
 | `--ingress-allowed-spiffe-ids IDS` | Comma-separated SPIFFE IDs (exact match) of the clients the ingress router lets reach actors; any listed client may reach any actor. Unset, no client may reach actors through the router. Both dataplanes require a Pod Identity CA client certificate on HTTPS and TLS CONNECT. The SPIFFE URI SAN allowlist may be checked during or after TLS, including after CONNECT acceptance, but unlisted clients must be denied before actor routing. The podidentity signer issues `spiffe://cluster.local/ns/<namespace>/sa/<service-account>`. `ATE_INGRESS_ALLOWED_SPIFFE_IDS` when the flag is absent |
+| `--egress-gateway-extra-spiffe-ids IDS` | Comma-separated SPIFFE IDs that hold ate-api-server's `egress_gateway` role next to atenet-egress, such as a credential provider that mints actor JWTs. ate-setup passes atenet-egress's ID and then each listed ID as repeated `--egress-gateway-spiffe-id` flags; unset passes none, leaving ate-api-server's default. `ATE_EGRESS_GATEWAY_EXTRA_SPIFFE_IDS` when the flag is absent |


The words here are confusing to me. "next to atenet-egress" - what does that mean? "In addition to" ?

Max Thompson (thompsonmax) changed the title ~~A credential plugin running as the credential-minter ServiceAccount can mint actor JWTs~~ ate-api-server's --egress-gateway-spiffe-id accepts a list of SPIFFE IDs Oct 28, 2026

Max Thompson (thompsonmax) force-pushed the credential-minter-role branch from baf6528 to a4eac92 Compare October 28, 2026 00:28

Max Thompson (thompsonmax) requested review from Taahir Ahmed (ahmedtd) and Tim Hockin (thockin) October 28, 2026 00:28

Max Thompson (thompsonmax) marked this pull request as ready for review October 28, 2026 00:29

Lior Lieberman (LiorLieberman) requested a review from Bowei Du (bowei) October 28, 2026 00:31

Taahir Ahmed (ahmedtd) added this to the M3 milestone Oct 28, 2026

Taahir Ahmed (ahmedtd) added the prio/P0 Highest priority / required for next milestone label Oct 28, 2026

Taahir Ahmed (ahmedtd) previously approved these changes Oct 28, 2026

View reviewed changes

Comment thread cmd/ateapi/main.go Outdated

Max Thompson (thompsonmax) dismissed Taahir Ahmed (ahmedtd)’s stale review via bf43a72 October 28, 2026 01:10

Max Thompson (thompsonmax) requested a review from Taahir Ahmed (ahmedtd) October 28, 2026 01:14

Shruti Nair (SHRUTI6991) reviewed Oct 28, 2026

View reviewed changes

Zheng Lammerts (zlammerts-svg) assigned Tim Hockin (thockin) Oct 28, 2026

Max Thompson (thompsonmax) mentioned this pull request Oct 28, 2026

ate-setup: let extra SPIFFE IDs hold the egress_gateway role #3393

Closed

2 tasks done

Max Thompson (thompsonmax) force-pushed the credential-minter-role branch from bf43a72 to 2eb56a4 Compare October 28, 2026 18:16

Max Thompson (thompsonmax) changed the title ~~ate-api-server's --egress-gateway-spiffe-id accepts a list of SPIFFE IDs~~ ate-api-server's --egress-gateway-spiffe-id is repeatable, and ate-setup sets it from ATE_EGRESS_GATEWAY_EXTRA_SPIFFE_IDS Oct 28, 2026

Max Thompson (thompsonmax) added 6 commits October 28, 2026 16:06

ateapi: take one egress gateway SPIFFE ID per flag

8594ebb

Make --egress-gateway-spiffe-id a StringArray so each value is one SPIFFE ID. StringSlice also split values on commas. Signed-off-by: Max Thompson <maxthomp@google.com>

ate-setup: share the ate-api-server flag rewrite

49ccb62

Move the arg rewrite out of setAPIServerAuthzArgs into setAPIServerFlag so other settings can render ate-api-server flags. Signed-off-by: Max Thompson <maxthomp@google.com>

Max Thompson (thompsonmax) force-pushed the credential-minter-role branch from 8e335d4 to b9afc9c Compare October 28, 2026 23:09

Tim Hockin (thockin) approved these changes Oct 29, 2026

View reviewed changes

Tim Hockin (thockin) added this pull request to the merge queue Oct 29, 2026

Merged via the queue into agent-substrate:main with commit eaf0e85 Oct 29, 2026
5 checks passed

Navigation Menu

Repository navigation

ate-api-server's --egress-gateway-spiffe-id is repeatable, and ate-setup sets it from ATE_EGRESS_GATEWAY_EXTRA_SPIFFE_IDS - #3360

ate-api-server's --egress-gateway-spiffe-id is repeatable, and ate-setup sets it from ATE_EGRESS_GATEWAY_EXTRA_SPIFFE_IDS#3360
Tim Hockin (thockin) merged 6 commits into
agent-substrate:mainfrom
thompsonmax:credential-minter-role

Max Thompson (thompsonmax) commented Oct 27, 2026 •

edited

Loading

Uh oh!

Uh oh!

Shruti Nair (SHRUTI6991) Oct 28, 2026 •

edited

Loading

Uh oh!

Max Thompson (thompsonmax) Oct 28, 2026 •

edited

Loading

Uh oh!

Tim Hockin (thockin) left a comment

Uh oh!

Tim Hockin (thockin) Oct 29, 2026

Uh oh!

Tim Hockin (thockin) Oct 29, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Max Thompson (thompsonmax) commented Oct 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Shruti Nair (SHRUTI6991) Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Max Thompson (thompsonmax) Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Tim Hockin (thockin) left a comment

Choose a reason for hiding this comment

Uh oh!

Tim Hockin (thockin) Oct 29, 2026

Choose a reason for hiding this comment

Uh oh!

Tim Hockin (thockin) Oct 29, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Max Thompson (thompsonmax) commented Oct 27, 2026 •

edited

Loading

Shruti Nair (SHRUTI6991) Oct 28, 2026 •

edited

Loading

Max Thompson (thompsonmax) Oct 28, 2026 •

edited

Loading