Skip to content

fix(substrate): let the golden actor fetch git and S3 skill sources - #3435

Open
danielle-mustillo wants to merge 5 commits into
mainfrom
danielle-mustillo/golden-egress-policy
Open

danielle-mustillo wants to merge 5 commits into
mainfrom
danielle-mustillo/golden-egress-policy

Conversation

@danielle-mustillo

@danielle-mustillo danielle-mustillo commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Why

Agents with skills from git or S3 never become Ready. They fail with GoldenActorCrashed.

The golden actor is the first copy of an Agent, and each session starts from its snapshot. The golden actor downloads git and S3 skills when it starts. It has no egress policy, so Substrate blocks the download. The Agent's egressPolicyRefs apply only to sessions.

We found this while testing the example Agents in #3417.

What changed

  • The ActorTemplate sets golden_egress_policy. The policy allows only the git and S3 hosts of the Agent's skills, including skills of its shared agents.
  • kagent install turns on Substrate's GoldenEgressPolicy preview gate, which Substrate needs to accept the policy.
  • If the gate is off, the Agent reports ActorTemplateRejected and names the gate. The controller keeps retrying, so the Agent becomes Ready after you turn on the gate. Other validation errors still stop retrying.

Agents without git or S3 skills don't change.

Example

An AgentTemplate with a git skill:

apiVersion: api.kagent.dev/v1alpha3
kind: AgentTemplate
metadata:
  name: <name>
spec:
  # ...
  skills:
    - name: review
      source:
        git:
          url: https://github.com/<org>/<repo>.git
          commit: <40-character commit SHA>

kagent now adds this to the compiled ActorTemplate:

golden_egress_policy:
  rules:
    - https:
        hostnames: [github.com]
        ports: {numbers: [443]}

The Agent's Ready condition:

Substrate Before After
GoldenEgressPolicy gate on False, GoldenActorCrashed True
GoldenEgressPolicy gate off False, GoldenActorCrashed False, ActorTemplateRejected, names the gate

Known limits

  • If you install Substrate yourself, add GoldenEgressPolicy to its Helm preview list.
  • S3 on a non-default port fails with 403 SignatureDoesNotMatch, because Substrate's egress gateway drops the port from the Host header.

Testing

On Kind with kagent install, Agents with GitHub skills and an S3 skill become Ready, and load_skill returns the skill. On main they fail with GoldenActorCrashed. With the gate off, the Agent shows the gate message, then becomes Ready once the gate is on.

@github-actions github-actions Bot added the bug Something isn't working label Oct 29, 2026
@danielle-mustillo
danielle-mustillo force-pushed the danielle-mustillo/golden-egress-policy branch from 7330005 to 98d9463 Compare October 29, 2026 06:17
…ources

The golden actor fetches git and S3 skill sources at startup, but its
ActorTemplate has no golden egress policy, so the fetch is blocked.

Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>
@danielle-mustillo
danielle-mustillo force-pushed the danielle-mustillo/golden-egress-policy branch from 98d9463 to 0d2e7c3 Compare October 29, 2026 20:20
Agents with git or S3 skill sources never became Ready. The golden actor
downloads those sources when it starts, but it had no egress policy, so
Substrate blocked the download.

The ActorTemplate now sets golden_egress_policy to allow only the hosts of
those sources, for the root agent and its shared agents. Sessions keep their
own egress policy. kagent install turns on Substrate's GoldenEgressPolicy
preview gate.

Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>
@danielle-mustillo
danielle-mustillo force-pushed the danielle-mustillo/golden-egress-policy branch from 0d2e7c3 to e2ab68d Compare October 29, 2026 20:35
@danielle-mustillo danielle-mustillo changed the title fix(substrate): give the golden actor an egress policy for git and S3 skill sources fix(substrate): let the golden actor fetch git and S3 skill sources Oct 29, 2026
@github-actions github-actions Bot added bug Something isn't working and removed bug Something isn't working labels Oct 29, 2026
@github-actions github-actions Bot added bug Something isn't working and removed bug Something isn't working labels Oct 29, 2026
… gate

Without Substrate's GoldenEgressPolicy preview gate, Substrate rejects a
template with a golden egress policy. Expect a message that names the
gate, and a retry that makes the Agent Ready once the gate is on. Other
validation errors on the same template must stay terminal.

Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>
…gate

Substrate rejects a golden egress policy while its GoldenEgressPolicy
preview gate is off, and reports golden_egress_policy as Forbidden. For
that error only, the Agent status names the gate and the controller
retries with backoff. The Agent becomes Ready after the gate is turned
on, with no controller restart. Other validation errors stay terminal.

Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>
Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>
@danielle-mustillo
danielle-mustillo force-pushed the danielle-mustillo/golden-egress-policy branch from e2ab68d to 8b91387 Compare October 29, 2026 21:28
@github-actions github-actions Bot added bug Something isn't working and removed bug Something isn't working labels Oct 29, 2026
@danielle-mustillo
danielle-mustillo marked this pull request as ready for review October 29, 2026 21:53
@danielle-mustillo
danielle-mustillo requested a review from a team as a code owner October 29, 2026 21:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant