diff --git a/cmd/ate-setup/commands.md b/cmd/ate-setup/commands.md index b10ce64d40..b551215364 100644 --- a/cmd/ate-setup/commands.md +++ b/cmd/ate-setup/commands.md @@ -32,6 +32,7 @@ a pre-scan pass, so they may appear anywhere on its command line. | `--cordon-system-infra-components` | `--cordon-system-infra-components` | Keep the system infra components, such as ate-api-server, atenet-router, atenet-egress and postgres, off the worker nodes. Assumes a small shared pool labeled and tainted `ate.dev/workloadType=ate-system-infra:NoSchedule`, across which each workload's replicas are spread, and a one-node pool labeled and tainted `ate.dev/workloadType=ate-postgres:NoSchedule` for postgres alone. `ATE_INSTALL_CORDON_SYSTEM_INFRA_COMPONENTS=true` when the flag is absent | | `--fake-data-plane` | `--fake-data-plane` | Install the fake data plane for control-plane benchmarks: fake-atelet takes atelet's place in its DaemonSet, taking its objectstore-plugin's storage settings, ate-controller is scaled to zero, and fake-workersync honors every WorkerPool with fake Workers on the fake-atelet nodes. Reaches `deploy ate-system`, `deploy atelet`, and `deploy ate-controller`; only `deploy ate-system` switches a cluster between the real and fake data planes, and the other two refuse a data plane other than the installed one. `deploy ate-system --fake-data-plane` refuses a cluster whose WorkerPools have worker pods. fake-atelet runs with its defaults; a benchmark tunes it through the optional `fake-atelet-tuning` ConfigMap, whose `FAKE_ATELET_*` variables set its flags, and which no install creates or resets. Actors do not exist; never use it on a cluster that serves real actors. `ATE_INSTALL_FAKE_DATA_PLANE=true` when the flag is absent | | `--ingress-allowed-spiffe-ids IDS` | `--ingress-allowed-spiffe-ids IDS`, or `ATE_INGRESS_ALLOWED_SPIFFE_IDS=IDS` | Comma-separated SPIFFE IDs (exact match) of the clients the ingress router lets reach actors; any listed client may reach any actor. Unset, no client may reach actors through the router. Both dataplanes require a Pod Identity CA client certificate on HTTPS and TLS CONNECT. The SPIFFE URI SAN allowlist may be checked during or after TLS, including after CONNECT acceptance, but unlisted clients must be denied before actor routing. The podidentity signer issues `spiffe://cluster.local/ns//sa/` | +| `--egress-gateway-extra-spiffe-ids IDS` | — | New. Comma-separated SPIFFE IDs that hold ate-api-server's `egress_gateway` role next to atenet-egress, such as a credential provider that mints actor JWTs. ate-setup passes atenet-egress's ID and then each listed ID as repeated `--egress-gateway-spiffe-id` flags; unset passes none, leaving ate-api-server's default. `ATE_EGRESS_GATEWAY_EXTRA_SPIFFE_IDS` when the flag is absent | | `--experimental-additional-egress-extproc-service NS/SVC:PORT` | `--experimental-additional-egress-extproc-service NS/SVC:PORT` | External processor authorization filter | | `--credential-provider JSON` | `--credential-provider JSON`, or `ATE_CREDENTIAL_PROVIDER` | Required by `deploy ate-system` and `deploy atenet`. A JSON object: `{"name":"k8s.io"}` deploys and uses the bundled Kubernetes Secrets provider, with a NetworkPolicy that admits only the egress gateway; `{"enabled":false}` turns egress credential injection off; `{"name":"","address":":"}` uses a provider you deploy yourself. See [`docs/egress-credential-injection.md`](../../docs/egress-credential-injection.md) | | `--ateapi-log-level LEVEL` | `--ateapi-log-level LEVEL`, or `ATE_API_LOG_LEVEL=LEVEL` | ate-api-server log level: `debug`, `info`, `warn`, or `error`. Written to the `ate-api-config` ConfigMap on every deploy when set; unset keeps the installed level (`info` on a new install). The server picks up a change without restarting | diff --git a/cmd/ate-setup/internal/config/config.go b/cmd/ate-setup/internal/config/config.go index 7a80966d89..d52c925f6f 100644 --- a/cmd/ate-setup/internal/config/config.go +++ b/cmd/ate-setup/internal/config/config.go @@ -121,6 +121,10 @@ type Config struct { // owner on global:root (ATE_API_AUTHZ_BOOTSTRAP_OWNERS). APIAuthzBootstrapOwners string + // EgressGatewayExtraSPIFFEIDs hold ate-api-server's egress_gateway role + // next to atenet-egress (ATE_EGRESS_GATEWAY_EXTRA_SPIFFE_IDS). + EgressGatewayExtraSPIFFEIDs []string + // ActorJWTAlgorithm is the signing algorithm of the key in a new actor JWT // pool (ACTOR_JWT_ALGORITHM): ES256 or RS256. ActorJWTAlgorithm string @@ -518,6 +522,10 @@ func (c *Config) ScriptEnv() []string { if len(c.IngressAllowedSPIFFEIDs) > 0 { merged["ATE_INGRESS_ALLOWED_SPIFFE_IDS"] = strings.Join(c.IngressAllowedSPIFFEIDs, ",") } + delete(merged, "ATE_EGRESS_GATEWAY_EXTRA_SPIFFE_IDS") + if len(c.EgressGatewayExtraSPIFFEIDs) > 0 { + merged["ATE_EGRESS_GATEWAY_EXTRA_SPIFFE_IDS"] = strings.Join(c.EgressGatewayExtraSPIFFEIDs, ",") + } if c.AdditionalEgressExtprocService != "" { merged["ATE_ADDITIONAL_EGRESS_EXTPROC_SERVICE"] = c.AdditionalEgressExtprocService } diff --git a/cmd/ate-setup/internal/config/config_test.go b/cmd/ate-setup/internal/config/config_test.go index b50366e1d9..3bf7be5662 100644 --- a/cmd/ate-setup/internal/config/config_test.go +++ b/cmd/ate-setup/internal/config/config_test.go @@ -61,6 +61,7 @@ func loadEnv(t *testing.T) { "ATE_API_POSTGRES_SERVER_CA_FILE", "ATE_ATENET_DATAPLANE", "ATE_CREDENTIAL_PROVIDER", + "ATE_EGRESS_GATEWAY_EXTRA_SPIFFE_IDS", "ATE_IMAGE_REPO", "ATE_IMAGE_TAG", "ATE_INGRESS_ALLOWED_SPIFFE_IDS", @@ -316,6 +317,44 @@ func TestLoadIngressAllowedSPIFFEIDs(t *testing.T) { } } +func TestLoadEgressGatewayExtraSPIFFEIDs(t *testing.T) { + const a, b = "spiffe://cluster.local/ns/a/sa/a", "spiffe://cluster.local/ns/b/sa/b" + for _, tc := range []struct { + name string + env string + want []string + wantErr string + }{ + {name: "unset", want: nil}, + {name: "several, with stray separators", env: " " + a + ",," + b + ", ", want: []string{a, b}}, + {name: "not a SPIFFE ID", env: a + ",https://example.com/x", wantErr: "ateapi.authz.egressGatewayExtraSPIFFEIDs must be a comma-separated list of SPIFFE IDs"}, + } { + t.Run(tc.name, func(t *testing.T) { + loadEnv(t) + if tc.env != "" { + t.Setenv("ATE_EGRESS_GATEWAY_EXTRA_SPIFFE_IDS", tc.env) + } + cfg, err := Load(Options{}) + if tc.wantErr != "" { + if err == nil || !strings.Contains(err.Error(), tc.wantErr) { + t.Fatalf("Load() error = %v, want one containing %q", err, tc.wantErr) + } + return + } + if err != nil { + t.Fatalf("Load() error = %v", err) + } + if !slices.Equal(cfg.EgressGatewayExtraSPIFFEIDs, tc.want) { + t.Errorf("EgressGatewayExtraSPIFFEIDs = %q, want %q", cfg.EgressGatewayExtraSPIFFEIDs, tc.want) + } + got, exported := scriptEnvMap(t, cfg)["ATE_EGRESS_GATEWAY_EXTRA_SPIFFE_IDS"] + if want := strings.Join(tc.want, ","); exported != (want != "") || got != want { + t.Errorf("ScriptEnv() ATE_EGRESS_GATEWAY_EXTRA_SPIFFE_IDS = %q (exported %v), want %q", got, exported, want) + } + }) + } +} + // The credential exchange impersonates the transfer account only on GCS, so // only a GKE deploy needs one, and its email needs both the name and the // project. diff --git a/cmd/ate-setup/internal/config/coverage_test.go b/cmd/ate-setup/internal/config/coverage_test.go index 8f23e55ea5..5585769f7a 100644 --- a/cmd/ate-setup/internal/config/coverage_test.go +++ b/cmd/ate-setup/internal/config/coverage_test.go @@ -90,7 +90,7 @@ func valueFor(key string) string { return RouterEnvoy case "clusterSize": return ClusterSizeSize0 - case "ingressAllowedSPIFFEIDs": + case "ingressAllowedSPIFFEIDs", "ateapi.authz.egressGatewayExtraSPIFFEIDs": return "spiffe://cluster.local/ns/a/sa/b" case "ateapi.postgres.cloudsql.ipType": return CloudSQLIPTypePSC diff --git a/cmd/ate-setup/internal/config/load.go b/cmd/ate-setup/internal/config/load.go index fa16bd5176..143efd52cc 100644 --- a/cmd/ate-setup/internal/config/load.go +++ b/cmd/ate-setup/internal/config/load.go @@ -230,7 +230,8 @@ func buildConfig(root string, env map[string]string, r *Resolved) (*Config, erro ClusterLocation: r.String("gcp.clusterLocation"), ExpectedJWTIssuer: r.String("ateapi.expectedJWTIssuer"), - APIAuthzBootstrapOwners: bootstrapOwners, + APIAuthzBootstrapOwners: bootstrapOwners, + EgressGatewayExtraSPIFFEIDs: splitList(r.String("ateapi.authz.egressGatewayExtraSPIFFEIDs")), BucketName: r.String("storage.bucketName"), SnapshotLocation: r.String("storage.snapshotLocation"), @@ -413,14 +414,20 @@ func validateResolved(cfg *Config, r *Resolved) error { return &InvalidError{Value: size, Want: ClusterSizeSize0 + " or " + ClusterSizeSize10} } - // The router rejects a malformed ID at startup; catch it before the - // rollout does. + // The router and ate-api-server reject a malformed ID at startup; catch it + // before the rollout does. for _, id := range cfg.IngressAllowedSPIFFEIDs { if _, err := spiffeid.FromString(id); err != nil { ids, _ := r.Value("ingressAllowedSPIFFEIDs") return &InvalidError{Value: ids, Want: fmt.Sprintf("a comma-separated list of SPIFFE IDs (%q: %v)", id, err)} } } + for _, id := range cfg.EgressGatewayExtraSPIFFEIDs { + if _, err := spiffeid.FromString(id); err != nil { + ids, _ := r.Value("ateapi.authz.egressGatewayExtraSPIFFEIDs") + return &InvalidError{Value: ids, Want: fmt.Sprintf("a comma-separated list of SPIFFE IDs (%q: %v)", id, err)} + } + } switch cfg.APILogLevel { case "", "debug", "info", "warn", "error": diff --git a/cmd/ate-setup/internal/config/setting.go b/cmd/ate-setup/internal/config/setting.go index 1fdea0effb..d13a1aaf1c 100644 --- a/cmd/ate-setup/internal/config/setting.go +++ b/cmd/ate-setup/internal/config/setting.go @@ -187,6 +187,13 @@ var Registry = []Setting{ Usage: "Comma-separated principals that always hold owner on global:root. " + "Unset means kubectl-ate's ServiceAccount, system:serviceaccount::ate-client", }, + { + Key: "ateapi.authz.egressGatewayExtraSPIFFEIDs", Env: "ATE_EGRESS_GATEWAY_EXTRA_SPIFFE_IDS", + Flag: "egress-gateway-extra-spiffe-ids", Kind: KindString, + Usage: "Comma-separated SPIFFE IDs that hold ate-api-server's egress_gateway role next to atenet-egress, " + + "such as a credential provider that mints actor JWTs. The role also reads actors and mints actor " + + "certificates. The podidentity signer issues spiffe://cluster.local/ns//sa/", + }, { // The egress gateway's credential provider, as a JSON object. Upstream // takes it as one value on one flag; it is carried the same way here diff --git a/cmd/ate-setup/internal/steps/apienv.go b/cmd/ate-setup/internal/steps/apienv.go index 481f72de1e..3cbf16a3cc 100644 --- a/cmd/ate-setup/internal/steps/apienv.go +++ b/cmd/ate-setup/internal/steps/apienv.go @@ -355,6 +355,9 @@ func (e *Env) applyWithAPIServerEnvHash(ctx context.Context, manifest []byte) er if err := setAPIServerAuthzArgs(objs, e.Cfg.APIAuthzBootstrapOwners); err != nil { return err } + if err := setAPIServerEgressGatewayArgs(objs, e.Namespace(), e.Cfg.EgressGatewayExtraSPIFFEIDs); err != nil { + return err + } if msg := sharedBootstrapOwnerWarning(e.Cfg.APIAuthzBootstrapOwners, e.Namespace()); msg != "" { log.Warnf("%s", msg) } @@ -365,9 +368,40 @@ func (e *Env) applyWithAPIServerEnvHash(ctx context.Context, manifest []byte) er const apiServerAuthzFlag = "--authz-bootstrap-owners" // setAPIServerAuthzArgs sets ate-api-server's bootstrap owners flag on its -// container in objs, replacing any the manifest already carries. A changed -// flag changes the pod template, so it rolls the Deployment. +// container in objs, replacing any the manifest already carries. func setAPIServerAuthzArgs(objs []*unstructured.Unstructured, bootstrapOwners string) error { + var values []string + if strings.Trim(bootstrapOwners, " ,") != "" { + values = []string{bootstrapOwners} + } + return setAPIServerFlag(objs, apiServerAuthzFlag, values) +} + +// apiServerEgressGatewayFlag is the ate-api-server flag +// setAPIServerEgressGatewayArgs owns. +const apiServerEgressGatewayFlag = "--egress-gateway-spiffe-id" + +// setAPIServerEgressGatewayArgs passes atenet-egress's SPIFFE ID and then each +// of extras, without duplicates, to ate-api-server's egress gateway flag. With +// no extras it drops the flag, leaving ate-api-server's default. +func setAPIServerEgressGatewayArgs(objs []*unstructured.Unstructured, namespace string, extras []string) error { + var ids []string + if len(extras) > 0 { + // Passing the flag replaces ate-api-server's default. + ids = []string{installdefaults.EgressSPIFFEID(namespace)} + for _, id := range extras { + if !slices.Contains(ids, id) { + ids = append(ids, id) + } + } + } + return setAPIServerFlag(objs, apiServerEgressGatewayFlag, ids) +} + +// setAPIServerFlag replaces every flag arg on ate-api-server's container in +// objs with one flag=value arg per value, in order. A changed flag changes the +// pod template, so it rolls the Deployment. +func setAPIServerFlag(objs []*unstructured.Unstructured, flag string, values []string) error { var dep *unstructured.Unstructured for _, obj := range objs { if obj.GetKind() == "Deployment" && obj.GetName() == "ate-api-server" { @@ -376,7 +410,7 @@ func setAPIServerAuthzArgs(objs []*unstructured.Unstructured, bootstrapOwners st } } if dep == nil { - return fmt.Errorf("the manifest has no deployment/ate-api-server to set authorization flags on") + return fmt.Errorf("the manifest has no deployment/ate-api-server to set %s on", flag) } containers, found, err := unstructured.NestedSlice(dep.Object, "spec", "template", "spec", "containers") if err != nil || !found { @@ -392,10 +426,10 @@ func setAPIServerAuthzArgs(objs []*unstructured.Unstructured, bootstrapOwners st return fmt.Errorf("%s: %w", kube.Describe(dep), err) } args = slices.DeleteFunc(args, func(arg string) bool { - return arg == apiServerAuthzFlag || strings.HasPrefix(arg, apiServerAuthzFlag+"=") + return arg == flag || strings.HasPrefix(arg, flag+"=") }) - if strings.Trim(bootstrapOwners, " ,") != "" { - args = append(args, apiServerAuthzFlag+"="+bootstrapOwners) + for _, v := range values { + args = append(args, flag+"="+v) } if err := unstructured.SetNestedStringSlice(container, args, "args"); err != nil { return fmt.Errorf("%s: %w", kube.Describe(dep), err) diff --git a/cmd/ate-setup/internal/steps/apienv_test.go b/cmd/ate-setup/internal/steps/apienv_test.go index 3c97dea3dc..db6694a706 100644 --- a/cmd/ate-setup/internal/steps/apienv_test.go +++ b/cmd/ate-setup/internal/steps/apienv_test.go @@ -695,6 +695,63 @@ spec: } } +func TestSetAPIServerEgressGatewayArgs(t *testing.T) { + const manifest = `apiVersion: apps/v1 +kind: Deployment +metadata: + name: ate-api-server + namespace: ate-system +spec: + template: + spec: + containers: + - name: ate-api-server + args: + - --config-file=/run/apiconfig.ate.dev/apiconfig.textproto + - --egress-gateway-spiffe-id=stale +` + const ( + egress = "spiffe://cluster.local/ns/ate-system/sa/atenet-egress" + a = "spiffe://cluster.local/ns/ate-system/sa/sts-credential-provider" + b = "spiffe://cluster.local/ns/plugins/sa/other" + ) + for _, tc := range []struct { + name string + extras []string + want []string + }{ + { + name: "gateway first, then extras without duplicates", + extras: []string{a, egress, b, a}, + want: []string{ + "--config-file=/run/apiconfig.ate.dev/apiconfig.textproto", + "--egress-gateway-spiffe-id=" + egress, + "--egress-gateway-spiffe-id=" + a, + "--egress-gateway-spiffe-id=" + b, + }, + }, + { + name: "no extras drops the flag", + want: []string{"--config-file=/run/apiconfig.ate.dev/apiconfig.textproto"}, + }, + } { + t.Run(tc.name, func(t *testing.T) { + objs, err := kube.DecodeManifestBytes([]byte(manifest)) + if err != nil { + t.Fatalf("DecodeManifestBytes() error = %v", err) + } + if err := setAPIServerEgressGatewayArgs(objs, "ate-system", tc.extras); err != nil { + t.Fatalf("setAPIServerEgressGatewayArgs() error = %v", err) + } + containers, _, _ := unstructured.NestedSlice(objs[0].Object, "spec", "template", "spec", "containers") + args, _, _ := unstructured.NestedStringSlice(containers[0].(map[string]any), "args") + if diff := cmp.Diff(tc.want, args); diff != "" { + t.Errorf("ate-api-server args (-want +got):\n%s", diff) + } + }) + } +} + func TestCreateAPIServerEnvVarsOmitsAuthz(t *testing.T) { e := &Env{Cfg: &config.Config{ TransferGSAName: testTransferGSAName, ProjectID: testProjectID, diff --git a/cmd/ateapi/internal/authz/authz_util.go b/cmd/ateapi/internal/authz/authz_util.go index a85729f4d7..1e1734d6cd 100644 --- a/cmd/ateapi/internal/authz/authz_util.go +++ b/cmd/ateapi/internal/authz/authz_util.go @@ -233,7 +233,7 @@ func EnsureStoreAndModel(ctx context.Context, pool *pgxpool.Pool, fgaServer *ser // every check, independent of any stored AccessPolicy. type SystemPrincipals struct { ControllerSPIFFEID string - EgressGatewaySPIFFEID string + EgressGatewaySPIFFEIDs []string IngressRouterSPIFFEID string AteletSPIFFEID string APIServerSidecarSPIFFEID string @@ -289,16 +289,22 @@ func parseBootstrapOwners(ids []string) (map[string]struct{}, error) { // parseSystemPrincipals validates system's SPIFFE IDs and maps each OpenFGA // user string to the system component roles it holds. func parseSystemPrincipals(system SystemPrincipals) (map[string][]string, error) { - bindings := []struct { + if len(system.EgressGatewaySPIFFEIDs) == 0 { + return nil, fmt.Errorf("no %s SPIFFE ID", RoleEgressGateway) + } + type binding struct { role string spiffeID string - }{ + } + bindings := []binding{ {role: RoleController, spiffeID: system.ControllerSPIFFEID}, - {role: RoleEgressGateway, spiffeID: system.EgressGatewaySPIFFEID}, {role: RoleIngressRouter, spiffeID: system.IngressRouterSPIFFEID}, {role: RoleAtelet, spiffeID: system.AteletSPIFFEID}, {role: RoleAPIServerSidecar, spiffeID: system.APIServerSidecarSPIFFEID}, } + for _, id := range system.EgressGatewaySPIFFEIDs { + bindings = append(bindings, binding{role: RoleEgressGateway, spiffeID: id}) + } out := make(map[string][]string, len(bindings)) for _, b := range bindings { if _, err := spiffeid.FromString(b.spiffeID); err != nil { diff --git a/cmd/ateapi/internal/authz/authz_util_test.go b/cmd/ateapi/internal/authz/authz_util_test.go index 77a13c26cf..da08a0887f 100644 --- a/cmd/ateapi/internal/authz/authz_util_test.go +++ b/cmd/ateapi/internal/authz/authz_util_test.go @@ -26,7 +26,7 @@ import ( func testSystemPrincipals() SystemPrincipals { return SystemPrincipals{ ControllerSPIFFEID: installdefaults.ControllerSPIFFEID(installdefaults.SystemNamespace), - EgressGatewaySPIFFEID: installdefaults.EgressSPIFFEID(installdefaults.SystemNamespace), + EgressGatewaySPIFFEIDs: []string{installdefaults.EgressSPIFFEID(installdefaults.SystemNamespace)}, IngressRouterSPIFFEID: installdefaults.RouterSPIFFEID(installdefaults.SystemNamespace), AteletSPIFFEID: installdefaults.AteletSPIFFEID(installdefaults.SystemNamespace), APIServerSidecarSPIFFEID: installdefaults.APIServerSPIFFEID(installdefaults.SystemNamespace), @@ -96,7 +96,7 @@ func TestParseSystemPrincipals(t *testing.T) { const router = "spiffe://cluster.local/ns/ate-system/sa/atenet-router" got, err := parseSystemPrincipals(SystemPrincipals{ ControllerSPIFFEID: "spiffe://cluster.local/ns/ate-system/sa/ate-controller", - EgressGatewaySPIFFEID: router, + EgressGatewaySPIFFEIDs: []string{router, "spiffe://cluster.local/ns/plugins/sa/token-exchange"}, IngressRouterSPIFFEID: router, AteletSPIFFEID: "spiffe://cluster.local/ns/ate-system/sa/atelet", APIServerSidecarSPIFFEID: "spiffe://cluster.local/ns/ate-system/sa/ate-api-server", @@ -106,7 +106,8 @@ func TestParseSystemPrincipals(t *testing.T) { } want := map[string][]string{ "user:spiffe%3A//cluster.local/ns/ate-system/sa/ate-controller": {RoleController}, - "user:spiffe%3A//cluster.local/ns/ate-system/sa/atenet-router": {RoleEgressGateway, RoleIngressRouter}, + "user:spiffe%3A//cluster.local/ns/ate-system/sa/atenet-router": {RoleIngressRouter, RoleEgressGateway}, + "user:spiffe%3A//cluster.local/ns/plugins/sa/token-exchange": {RoleEgressGateway}, "user:spiffe%3A//cluster.local/ns/ate-system/sa/atelet": {RoleAtelet}, "user:spiffe%3A//cluster.local/ns/ate-system/sa/ate-api-server": {RoleAPIServerSidecar}, } @@ -123,13 +124,16 @@ func TestNewRejectsInvalidSystemPrincipals(t *testing.T) { wantErr string }{ {"empty controller SPIFFE ID", func(p *SystemPrincipals) { p.ControllerSPIFFEID = "" }, "invalid controller SPIFFE ID"}, - {"empty egress gateway SPIFFE ID", func(p *SystemPrincipals) { p.EgressGatewaySPIFFEID = "" }, "invalid egress_gateway SPIFFE ID"}, + {"no egress gateway SPIFFE IDs", func(p *SystemPrincipals) { p.EgressGatewaySPIFFEIDs = nil }, "no egress_gateway SPIFFE ID"}, + {"empty egress gateway SPIFFE ID in list", func(p *SystemPrincipals) { + p.EgressGatewaySPIFFEIDs = []string{defaults.EgressGatewaySPIFFEIDs[0], ""} + }, "invalid egress_gateway SPIFFE ID"}, {"empty ingress router SPIFFE ID", func(p *SystemPrincipals) { p.IngressRouterSPIFFEID = "" }, "invalid ingress_router SPIFFE ID"}, {"empty atelet SPIFFE ID", func(p *SystemPrincipals) { p.AteletSPIFFEID = "" }, "invalid atelet SPIFFE ID"}, {"empty api server sidecar SPIFFE ID", func(p *SystemPrincipals) { p.APIServerSidecarSPIFFEID = "" }, "invalid api_server_sidecar SPIFFE ID"}, {"bare ServiceAccount name", func(p *SystemPrincipals) { p.ControllerSPIFFEID = "ate-controller" }, "invalid controller SPIFFE ID"}, {"non-spiffe URI", func(p *SystemPrincipals) { - p.EgressGatewaySPIFFEID = "https://cluster.local/ns/ate-system/sa/atenet-egress" + p.EgressGatewaySPIFFEIDs = []string{"https://cluster.local/ns/ate-system/sa/atenet-egress"} }, "invalid egress_gateway SPIFFE ID"}, } { t.Run(tc.name, func(t *testing.T) { diff --git a/cmd/ateapi/internal/authz/interceptor_test.go b/cmd/ateapi/internal/authz/interceptor_test.go index 110ef6bf1d..b20bd6a7a6 100644 --- a/cmd/ateapi/internal/authz/interceptor_test.go +++ b/cmd/ateapi/internal/authz/interceptor_test.go @@ -626,7 +626,7 @@ func TestUnaryServerInterceptor_SystemPrincipals(t *testing.T) { ) authorizer, _, err := New(ctx, pool, fgaServer, nil, SystemPrincipals{ ControllerSPIFFEID: controllerID, - EgressGatewaySPIFFEID: egressID, + EgressGatewaySPIFFEIDs: []string{egressID}, IngressRouterSPIFFEID: routerID, AteletSPIFFEID: ateletID, APIServerSidecarSPIFFEID: pluginID, diff --git a/cmd/ateapi/internal/store/atepg/common_test.go b/cmd/ateapi/internal/store/atepg/common_test.go index 30a21388ba..c25a3ecbfa 100644 --- a/cmd/ateapi/internal/store/atepg/common_test.go +++ b/cmd/ateapi/internal/store/atepg/common_test.go @@ -57,7 +57,7 @@ func setTestPolicyManager(t *testing.T, p *Persistence) { t.Cleanup(fgaServer.Close) _, policyManager, err := authz.New(t.Context(), p.pool, fgaServer, nil, authz.SystemPrincipals{ ControllerSPIFFEID: installdefaults.ControllerSPIFFEID(installdefaults.SystemNamespace), - EgressGatewaySPIFFEID: installdefaults.EgressSPIFFEID(installdefaults.SystemNamespace), + EgressGatewaySPIFFEIDs: []string{installdefaults.EgressSPIFFEID(installdefaults.SystemNamespace)}, IngressRouterSPIFFEID: installdefaults.RouterSPIFFEID(installdefaults.SystemNamespace), AteletSPIFFEID: installdefaults.AteletSPIFFEID(installdefaults.SystemNamespace), APIServerSidecarSPIFFEID: installdefaults.APIServerSPIFFEID(installdefaults.SystemNamespace), diff --git a/cmd/ateapi/internal/store/storetest/storetest.go b/cmd/ateapi/internal/store/storetest/storetest.go index 7bc219b3d2..2d0f4a5dff 100644 --- a/cmd/ateapi/internal/store/storetest/storetest.go +++ b/cmd/ateapi/internal/store/storetest/storetest.go @@ -101,7 +101,7 @@ func SetupPostgresPersistence(t *testing.T) *atepg.Persistence { func DefaultSystemPrincipals() authz.SystemPrincipals { return authz.SystemPrincipals{ ControllerSPIFFEID: installdefaults.ControllerSPIFFEID(installdefaults.SystemNamespace), - EgressGatewaySPIFFEID: installdefaults.EgressSPIFFEID(installdefaults.SystemNamespace), + EgressGatewaySPIFFEIDs: []string{installdefaults.EgressSPIFFEID(installdefaults.SystemNamespace)}, IngressRouterSPIFFEID: installdefaults.RouterSPIFFEID(installdefaults.SystemNamespace), AteletSPIFFEID: installdefaults.AteletSPIFFEID(installdefaults.SystemNamespace), APIServerSidecarSPIFFEID: installdefaults.APIServerSPIFFEID(installdefaults.SystemNamespace), diff --git a/cmd/ateapi/main.go b/cmd/ateapi/main.go index 8e9d74d6f8..9ade6ee253 100644 --- a/cmd/ateapi/main.go +++ b/cmd/ateapi/main.go @@ -94,7 +94,7 @@ var ( // These SPIFFE IDs hold the matching system component role when they call // over mTLS. controllerSPIFFEID = pflag.String("controller-spiffe-id", installdefaults.ControllerSPIFFEID(installdefaults.NamespaceFromPodEnv()), "mTLS SPIFFE ID of ate-controller. Holds the controller authz role, which manages workers.") - egressGatewaySPIFFEID = pflag.String("egress-gateway-spiffe-id", installdefaults.EgressSPIFFEID(installdefaults.NamespaceFromPodEnv()), "mTLS SPIFFE ID of atenet-egress. Holds the egress_gateway authz role, which reads and lists actors and reads their egress policies.") + egressGatewaySPIFFEIDs = pflag.StringArray("egress-gateway-spiffe-id", []string{installdefaults.EgressSPIFFEID(installdefaults.NamespaceFromPodEnv())}, "mTLS SPIFFE IDs of atenet-egress and any other client that holds the egress_gateway authz role, which reads and lists actors and reads their egress policies. Repeat the flag for each ID.") ingressRouterSPIFFEID = pflag.String("ingress-router-spiffe-id", installdefaults.RouterSPIFFEID(installdefaults.NamespaceFromPodEnv()), "mTLS SPIFFE ID of atenet-router. Holds the ingress_router authz role, which resumes and lists actors.") apiServerSidecarSPIFFEID = pflag.String("api-server-sidecar-spiffe-id", installdefaults.APIServerSPIFFEID(installdefaults.NamespaceFromPodEnv()), "mTLS SPIFFE ID of the ate-api-server pod, which its sidecars share. Holds the api_server_sidecar role, which reads actors so the objectstore control plugin can scope their storage credentials.") @@ -257,7 +257,7 @@ func main() { defer fgaServer.Close() authorizer, policyManager, err := authz.New(shutdownCtx, pool, fgaServer, *authzBootstrapOwners, authz.SystemPrincipals{ ControllerSPIFFEID: *controllerSPIFFEID, - EgressGatewaySPIFFEID: *egressGatewaySPIFFEID, + EgressGatewaySPIFFEIDs: *egressGatewaySPIFFEIDs, IngressRouterSPIFFEID: *ingressRouterSPIFFEID, AteletSPIFFEID: ateletSPIFFEID, APIServerSidecarSPIFFEID: *apiServerSidecarSPIFFEID, diff --git a/docs/authorization.md b/docs/authorization.md index ab5cb19cd5..7879449a21 100644 --- a/docs/authorization.md +++ b/docs/authorization.md @@ -135,7 +135,7 @@ through their pod certificates. `ate-api-server`'s `--controller-spiffe-id`, | Role | Held by | Allows | |---|---|---| | `controller` | `ate-controller` | Create, get, list, update, delete and drain workers | -| `egress_gateway` | `atenet-egress` | Get actors and their egress policies; list actors across all atespaces; mint actor JWTs and certificates | +| `egress_gateway` | `atenet-egress`, and any other SPIFFE IDs passed with `--egress-gateway-spiffe-id`, which ate-setup sets from `ATE_EGRESS_GATEWAY_EXTRA_SPIFFE_IDS` | Get actors and their egress policies; list actors across all atespaces; mint actor JWTs and certificates | | `ingress_router` | `atenet-router` | Resume actors; list actors across all atespaces | | `atelet` | `atelet` | Get actors' egress policies, for the worker that enforces them | | `api_server_sidecar` | the `ate-api-server` pod identity, which its sidecars share, today the objectstore control plugin | Get actors, to derive the snapshot prefixes a per-actor storage credential may cover | diff --git a/docs/operator-install.md b/docs/operator-install.md index 15b649a8a3..a9c1cd4f7e 100644 --- a/docs/operator-install.md +++ b/docs/operator-install.md @@ -277,6 +277,7 @@ demos, the [benchmarks](../benchmarking/README.md#settings), and the | `actorJWT.algorithm` | `ACTOR_JWT_ALGORITHM` | `--actor-jwt-algorithm` | `ES256` | (global) | | `actorJWT.issuer` | `ACTOR_JWT_ISSUER` | `--actor-jwt-issuer` | — | (global) | | `ateapi.authz.bootstrapOwners` | `ATE_API_AUTHZ_BOOTSTRAP_OWNERS` | `--ateapi-authz-bootstrap-owners` | `system:serviceaccount::ate-client` | (global) | +| `ateapi.authz.egressGatewayExtraSPIFFEIDs` | `ATE_EGRESS_GATEWAY_EXTRA_SPIFFE_IDS` | `--egress-gateway-extra-spiffe-ids` | — | (global) | | `ateapi.expectedJWTIssuer` | `EXPECTED_JWT_ISSUER` | `--ateapi-expected-jwt-issuer` | — | (global) | | `ateapi.logLevel` | `ATE_API_LOG_LEVEL` | `--ateapi-log-level` | — | (global) | | `ateapi.postgres.cloudsql.gsa` | `ATE_API_POSTGRES_CLOUDSQL_GSA` | `--ateapi-postgres-cloudsql-gsa` | — | (global) |