Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,6 @@ Thumbs.db
# root; tools with their own go.mod land beside their source. Build with
# `-o bin/<tool>`, which is ignored above.
/validate-image-cache
/setup-gcp
/tools/apitool/apitool
/tools/junittool/junittool

Expand Down
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ It takes the Kubernetes control-plane out of the critical path to achieve lower
Agent Substrate relies on the fact that agent-like applications tend to be idle most of the time to achieve heavy multiplexing.

For development, it's recommended to read the `README.md` and `CONTRIBUTING.md` in the root folder.
See `cmd/ate-setup` and `tools/setup-gcp` for provisioning and deploying clusters and GCP resources.
See `cmd/ate-setup` for deploying clusters. GCP resources are provisioned with `tools/setup-gcp` in [substrate-gke](https://github.com/ai-on-gke/substrate-gke).

## Repository Layout

Expand Down Expand Up @@ -156,7 +156,7 @@ See the [metric registry](docs/observability.md#the-metric-registry) section of
1. Write tests for new behavior. Update tests when you change existing behavior. Consider the value of tests for future changes.
2. Ensure changes do not break existing tests.
3. Run `go test -race ./...` and `hack/verify-all.sh` locally before requesting a code review to catch common issues like missed copyright headers or formatting drift.
4. For end-to-end tests involving the actual infrastructure, ensure you have a running cluster (setup via `hack/ate-dev-env.sh.example` and `go run ./tools/setup-gcp bootstrap`).
4. For end-to-end tests involving the actual infrastructure, ensure you have a running cluster (on GKE, set up with `tools/setup-gcp` in [substrate-gke](https://github.com/ai-on-gke/substrate-gke)).
5. A test that skips when a precondition is missing (Docker, a cluster, an artifact directory) must **fail** on it in CI, because a skip and a pass are the same exit code. Check for CI with a named predicate like `cmd/ateapi/internal/store/dockerenv.Required()` instead of an inline `os.Getenv("CI")`. Make sure the CI branch actually fails before merging. See `docs/dev/best-practices/ci-fail-closed.md`.

## Security Considerations
Expand Down
50 changes: 19 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,19 +167,18 @@ curl -X POST -i http://localhost:8000/
gcloud auth application-default login --project=${PROJECT_ID}
```

3. Provision the required GCP resources (GKE cluster, GCS, and IAM bindings):
```bash
go run ./tools/setup-gcp bootstrap
```

On a fresh project this step also creates the Workload Identity IAM grants
that ate-api-server and atelet depend on. See
[what `create iam` actually grants](tools/setup-gcp/README.md#what-create-iam-actually-grants)
to audit them or apply them manually. If you bring your own cluster instead,
it needs the PodCertificateRequest and ClusterTrustBundle APIs, which GKE
1.37 and higher serve by default. On 1.36, enable them as beta APIs, then
replace any 1.36 node pools created before that. See the
[Create Cluster warning](tools/setup-gcp/README.md#2-create-cluster).
3. Provision the required GCP resources (GKE cluster, GCS, and IAM bindings)
with [`tools/setup-gcp`](https://github.com/ai-on-gke/substrate-gke/tree/main/tools/setup-gcp) in the
[substrate-gke](https://github.com/ai-on-gke/substrate-gke) repository, which holds the GKE
setup and teardown tooling. Its `bootstrap` command also creates the Workload
Identity IAM grants that ate-api-server and atelet depend on. See
[what `create iam` actually grants](https://github.com/ai-on-gke/substrate-gke/blob/main/tools/setup-gcp/README.md#what-create-iam-actually-grants)
to audit them or apply them manually.

If you bring your own cluster instead, it needs the PodCertificateRequest and
ClusterTrustBundle APIs, which GKE 1.37 and higher serve by default. On 1.36,
enable them as beta APIs, then replace any 1.36 node pools created before
that. See the [Create Cluster warning](https://github.com/ai-on-gke/substrate-gke/blob/main/tools/setup-gcp/README.md#2-create-cluster).

4. Deploy the Agent Substrate system to your cluster:
```bash
Expand Down Expand Up @@ -210,17 +209,11 @@ curl -X POST -i http://localhost:8000/

#### Custom Setup and Deployment

You can run individual setup steps to create GCP resources as needed. See `go run ./tools/setup-gcp --help` for available options. For example:
```bash
go run ./tools/setup-gcp create cluster
go run ./tools/setup-gcp create bucket
```

To run the PostgreSQL store backend on Cloud SQL — with IAM database
authentication and no passwords — see [tools/setup-gcp/cloud-sql.md](tools/setup-gcp/cloud-sql.md).
To create individual GCP resources, or to run the PostgreSQL store backend on
Cloud SQL, see [`tools/setup-gcp`](https://github.com/ai-on-gke/substrate-gke/tree/main/tools/setup-gcp) in substrate-gke.

Similarly, you can deploy or clean up specific Agent Substrate components with
`ate-setup`. See `go run ./cmd/ate-setup --help` for all options.
You can deploy or cleanup specific Agent Substrate components using
the installation script. See `go run ./cmd/ate-setup` for all options.
```bash
# Re-deploy only ate-apiserver of the ATE system
go run ./cmd/ate-setup deploy apiserver
Expand All @@ -231,17 +224,13 @@ go run ./cmd/ate-setup delete all

#### Tearing down resources (GCP)

If you need to delete the resources created by the setup script, you can use the provided script `hack/teardown.sh`. This script will delete resources in the reverse order of creation and handles partial failures gracefully.

```bash
./hack/teardown.sh [--all|--cluster-scoped]
```
To delete the GCP resources, run substrate-gke's
[`tools/setup-gcp/teardown.sh`](https://github.com/ai-on-gke/substrate-gke/blob/main/tools/setup-gcp/README.md#teardown). It deletes resources
Comment thread
stshrive marked this conversation as resolved.
in the reverse order of creation and handles partial failures.

* `--all` deletes everything, including the dashboards, the snapshot transfer service account and the project-level IAM bindings, which every Substrate cluster in the project shares.
* `--cluster-scoped` deletes only this cluster, its buckets and the bindings on them. Use it while other Substrate clusters still use the project.

Or run individual teardown steps as needed (see `./hack/teardown.sh` for available options).

#### Tearing down local `kind` resources

If you need to delete the local `kind` cluster and its registry (if it was created by `hack/create-kind-cluster.sh`):
Expand Down Expand Up @@ -293,5 +282,4 @@ We provide several sample applications demonstrating Agent Substrate's capabilit
will eventually ship in upstream Kubernetes (with different names).
* `cmd/kubectl-ate`: A CLI tool for managing Agent Substrate resources. See its [README](cmd/kubectl-ate/README.md).
* `cmd/benchmarking`: Synthetic workloads used by the load tests, including `glutton`, which consumes RAM, disk, and file descriptors on demand.
* `tools/setup-gcp`: A provisioning utility to set up the necessary GCP infrastructure resources (GKE, GCS, IAM).
* `demos/`: Sample applications demonstrating Agent Substrate capabilities.
202 changes: 0 additions & 202 deletions _LICENSES/cloud.google.com/go/container/LICENSE

This file was deleted.

Loading
Loading