diff --git a/.agents/skills/agent-release-gate/SKILL.md b/.agents/skills/agent-release-gate/SKILL.md index 49372ec5cc..aa01fe2a51 100644 --- a/.agents/skills/agent-release-gate/SKILL.md +++ b/.agents/skills/agent-release-gate/SKILL.md @@ -344,10 +344,13 @@ ever reaches the stream. An empty ledger FAILS a cell; missing evidence is not e - `resources/matrix_l1_lifecycle_routes.py` — **MANDATORY. [mechanism-blind]** the routing matrix itself: for each kind of mid-conversation config change, assert the route the runner took. One - sandbox id = applied in place, two = rebuilt. Blocks on the four unambiguous cases (no change - must stay warm; an instructions edit, a permissions edit and a tool-catalog edit must escalate) - and reports the `model` case rather than guessing at a deployment's connection shape. This is - the cell that would have caught the `cold1` rot described below. + sandbox id = applied in place, two = rebuilt. Blocks on six cases: no change must stay warm; an + instructions edit, a permissions edit and a tool-catalog edit must escalate; and a + same-connection model switch must stay warm on BOTH claude and pi_core. The pi_core model case + (added 2026-08-29) is the standing trap for the wire-spelling bug class: the router once keyed + its table on the bare "pi" literal while the wire carries "pi_core", every playground model + switch silently rebuilt, and the claude-only case could not see it (#6364). This is the cell + that would have caught the `cold1` rot described below. - `resources/matrix_l2_approval_across_config_change.py` — **MANDATORY. [coached]** the killer combination: an approval answered while a config change rides along in the SAME request. It is the regression test for the applied-state bug (the pool used to stamp the INCOMING fingerprint @@ -429,6 +432,82 @@ lands on a pool miss and takes the cold decision-map path, which is exactly the live Gmail and GitHub Composio connections in the target project; skip it otherwise. - `resources/seeds/` — representative green `results.json` files kept as regression-seed references. +### The incident checks — born from the free-credits 401 of 2026-08-30 + +A free-credits user on cloud hit a 401 because a fresh Daytona sandbox's first model call raced +the asynchronous substitution of its Daytona Secret: the provider got the raw `dtn_secret_` +placeholder. The product then blamed the user's own key, which was wrong. The same release fixed a +family of warm-session over-evictions caused by drift between two identity views in the runner. +These four checks make each layer's failure loud instead of silent. Run all four on every gate. + +- `resources/matrix_c5_first_call_race.py` — **[mechanical]** the placeholder race, and whether it + is reported honestly. Mints a new workflow so the sandbox is necessarily cold, sends one short + message so the first model call lands as early as possible, and asserts the STORED turn row came + back. PASSes when the turn succeeds or when the failure carries the runner's + `credential_delivery_failed` code with its retry copy. FAILs when the run advises adding a key + while the underlying refusal carries the placeholder signature (`Received=dtn_`/`dtn_secret_`) — + the incident itself. The assertion is deliberately body-INDEPENDENT: only the litellm proxy + echoes a placeholder, so on a direct provider (where BYO-key cloud users live) an echo test is + blind, and F6 shipped a user-blaming 401 straight through the first version of this cell. A + credential refusal on this cell's necessarily-fresh sandbox must never advise adding a key, echo + or no echo; with PR #6408 the honest classification is `credential_delivery_failed`. Against a + deployment predating #6408 that assertion fails by construction — pass `--pre-6408` to report it + as a SKIP naming the known gap instead of an unexplained failure. It also counts `Received=dtn_` + lines in the credits proxy and reports the + count as diagnostic, never as a verdict. The proxy is never guessed by name across the box: it + must be named with `--proxy-container`, or belong to the target stack's compose project + (`--compose-project`, else derived from whichever container publishes the port in + `AGENTA_BASE`). With no match it prints "no credits proxy in this deployment; count not + applicable" and carries on. Reading a foreign project's proxy invents evidence about a + deployment that was never under test, which is worse than reading none. A run that dies on an + exhausted provider key SKIPs with "environment: provider key out of credit" rather than + failing — but only when the stored error carries a credit or billing signature, and never + when a placeholder refusal is present, because that combination is the incident itself. +- `resources/sweep_disagree.py` — **[mechanism-level invariant; run AFTER a gate session]** greps + the runner log for `[reconcile] shadow ... DISAGREE ...`, the line `logReconcileShadow` writes + when the coordinator's `configFingerprint` decision and the router's facet digests disagree. + That drift is the over-eviction signature and it is invisible from the wire — the turn still + succeeds, it just paid for a rebuild it did not need — so a log sweep is the only way to catch + it. `--since ` is required; `--container` defaults to autodetecting the local + stack's runner. Exits 0 PASS, 1 FAIL (printing the offending lines), 2 SKIP when the log is not + reachable. Three line shapes are excluded as known SHADOW-COMPARATOR gaps (triage 2026-08-31, + `f7-disagree-triage.md`): the coordinator is correct and pinned, only the shadow's model of it + disagrees, and the comparator fixes are a post-release follow-up — without the exceptions the + sweep fails on the runner's own expected behavior on every loaded window. They are never + silent: each excluded line is printed with its shape and the triage marker, the excluded count + is reported separately, each shape is anchored on both halves of the line so it cannot swallow + a real disagreement, and any line matching no shape still FAILS. Delete a shape when its fix + lands; `--no-exceptions` fails on every DISAGREE line and is how you prove one can go. +- `resources/matrix_h1_bad_harness.py` — **[mechanical]** a malformed harness must fail closed. + Drives three unreadable `harness` blocks (a wrong-type value, an unknown string, a null kind) at + both the commit API and the live invoke, and records WHICH boundary refused (`commit_api`, + `invoke_http`, or `runner_stream`) rather than demanding a particular one — a refusal further + out is better, not worse. The invariant is that some boundary refuses attributably and no turn + ever runs on a defaulted harness. FAILs if a turn executes and stores output. +- `resources/check_secrets_teardown.py` — **[mechanical]** a Daytona Secret must not outlive its + run. Inventories the Daytona organization's Secret NAMES (never values) before a short Daytona + journey, forces the teardown with a config-change eviction, then asserts every `agenta_*` Secret + the run created is gone within a bounded settle window. Needs a Daytona API key in the + environment (`DAYTONA_API_KEY` or `AGENTA_RUNNER_DAYTONA_API_KEY`) and SKIPs with the exact + reason without one. Run it alone: a concurrent Daytona run against the same organization looks + the same as a leftover. The listing walks `GET /secret/paginated` by cursor to exhaustion — + `/secrets` does not exist, plain `/secret` is deprecated and fails above 1500 secrets, and a + `page` parameter is silently ignored, so anything less than real cursor pagination is noise + against an organization this size. The settle loop polls `GET /secret/{secretId}` per created + Secret rather than re-enumerating. `test_check_secrets_teardown_pagination.py` pins the walk. + A journey that dies on an exhausted provider key never creates a Secret, so it SKIPs with + "environment: provider key out of credit" instead of failing the teardown path it never + exercised. +- `qa_matrix_lib.out_of_credit(error_text, codes)` — **[shared classification; no cell of its + own]** the SKIP reason when a run failed ONLY because the provider key has no credit left, and + `None` for everything else. An exhausted key is an environment condition: a cell that renders + it as FAIL spends a reviewer's attention on a topped-up balance, and teaches the reader that + this cell's FAIL is sometimes noise, which is how a real regression gets waved through later. + Recognition is narrow in both directions — the `starter_credits_*` codes plus the runner's own + credits copy and the provider's billing refusal, and deliberately NOT a bare 401, a rate limit, + or the placeholder refusal. Wired into `matrix_c5_first_call_race.py` and + `check_secrets_teardown.py`; `test_out_of_credit_skip.py` pins the boundary from both sides. + ## Contributing Before committing any resource script, run the repo-pinned ruff (`uv run --no-sync ruff format` diff --git a/.agents/skills/agent-release-gate/resources/LESSONS.md b/.agents/skills/agent-release-gate/resources/LESSONS.md index 88e1a62a04..226ecf8ce7 100644 --- a/.agents/skills/agent-release-gate/resources/LESSONS.md +++ b/.agents/skills/agent-release-gate/resources/LESSONS.md @@ -228,6 +228,100 @@ around — `CODEX_SQLITE_HOME` is split onto container-local disk) and hard link the API reads S3 directly, so a hit is store-side proof, and the same listing shows any 0-byte objects — the fingerprint of this whole bug class. +## Two Daytona-era traps for the lifecycle (L*) cases — 2026-08-31 + +**Fixture connections must be vault-backed.** A fixture built by copying the Claude defaults and +changing only the harness keeps `llm.connection = {"mode":"self_managed","slug":null}`. A +self_managed Pi run needs the `PI_CODING_AGENT_DIR` mount, which a gate deployment does not have, +so turn 1 errors and the case fails before it tests anything. Set +`llm.connection = {"mode":"agenta","slug":null}` on every non-default-harness fixture (caught on +#6371; the Pi fixtures in `matrix_l5_live_route_observed.py` and `bench_lib.py` already do this). + +**A stuck-substitution rebuild is not an eviction.** Since the credential preflight (#6370), a +fresh Daytona sandbox whose Secret wiring failed (a vendor-side per-sandbox fault, a few percent +of creates) is convicted at ~10s and rebuilt ONCE. A warm-reuse case that counts sandbox ids can +therefore see two ids without any lifecycle regression. Before ruling a warm case failed, grep +the runner log for `[credential-preflight] STUCK`: if it fired inside the run, re-run the case +instead of reporting the eviction. + +## On a placeholder 401, read the proxy log before you blame a key — 2026-08-31 + +A free-credits user on cloud hit a 401 on their first message. The product told them to add the +project's OpenAI key. That advice was wrong three ways: their key was fine, adding one would not +have helped, and the run was retryable. The real cause was the first-call race. On a Daytona run +the real key never enters the sandbox — it is a Daytona Secret, and the sandbox holds a +`dtn_secret_` placeholder that Daytona substitutes into egress asynchronously, with no +confirmation signal, 10-24s after the Secret is created. A cold sandbox whose FIRST model call +beats that propagation sends the raw placeholder, and the provider refuses it with a 401. + +**The rule.** A 401 from a Daytona run is not evidence about the user's key until you have read +the litellm-proxy log. Grep it for `Received=dtn_`. If the line is there, the key was never the +problem and no key change will fix it: the run needed a retry. + +**An ABSENT marker proves nothing.** The marker is one-directional evidence — present, it +confirms a placeholder refusal; absent, it is silence, and silence has many causes. A direct +provider never emits it at all (`api.anthropic.com` answers "Invalid bearer token" and echoes +nothing), a remote deployment has no reachable proxy log, and incomplete log access looks +identical to a clean window. Reading an empty grep as "so it really was the user's key" is how F6 +survived a whole release. When the marker is absent, judge on the other evidence instead: the +stored error's CODE (`credential_delivery_failed` is the runner's own verdict and outranks any +grep), whether the sandbox was freshly created, and whether the copy contradicts itself by +advising a key change on a run whose key was delivered seconds earlier. The runner classifies +this correctly as `credential_delivery_failed` +(see `PLACEHOLDER_CREDENTIAL` in `services/runner/src/engines/sandbox_agent/errors.ts`), so a +run that reports an add-a-key message over a placeholder refusal is a product bug, not a user +error. The related trap already recorded above still holds: a stuck-substitution rebuild is not an +eviction, so grep `[credential-preflight] STUCK` before calling a warm case failed. + +Four standing checks came out of this incident. Run all four on every gate; each one is described +in full in the skill's resource inventory. + +1. `matrix_c5_first_call_race.py` — forces a cold Daytona sandbox and sends one message + immediately, so the first model call lands as early as it can. FAILs when a placeholder refusal + is reported as the user's key problem. +2. `sweep_disagree.py` — run AFTER a gate session. Greps the runner log for + `[reconcile] shadow ... DISAGREE ...`, the over-eviction signature that never shows on the wire. +3. `matrix_h1_bad_harness.py` — a malformed harness must be refused at some boundary and must + never run as a silent defaulted turn. +4. `check_secrets_teardown.py` — a Daytona Secret must not outlive its run. Names only, never + values. + +## Two traps the incident checks hit on their first live run — 2026-08-31 + +**The Daytona Secrets API is singular, paginated, and lies about `page`.** `/secrets` does not +exist; it 404s with "Cannot GET". The real paths are `/secret`, `/secret/paginated` and +`/secret/{secretId}` (verified against `@daytona/api-client@0.198.0` inside the runner). Plain +`/secret` is deprecated and, per the client's own docs, "fails for organizations with more than +1500 secrets" — and the org holds ~3510, so it is unusable. The paginated listing returns 100 per +response and a `page` parameter is SILENTLY IGNORED: the same 100 ids come back every time, which +makes a page-based walk loop forever on identical data while looking like progress. Follow +`nextCursor` to exhaustion, refuse a cursor that repeats, and bound the walk. A useful side +effect: because `/secret` and `/secret/paginated` return 403 for an under-scoped key while +`/secrets` returns 404, you can confirm the right path without any list access at all. + +**Never pick a container by name match on a shared box.** `matrix_c5` originally took the first +`docker ps` name containing "litellm" and found `starter-litellm-proxy` — a different project's +container — while the stack under test had no proxy at all. A foreign container's log is worse +than no log: it invents evidence about a deployment that was never under test. Resolve a +container by its `com.docker.compose.project` label against the target stack's project (derive +the project from whichever container publishes the port in `AGENTA_BASE`), or take it explicitly. +When nothing matches, say so and continue. + +## Standing reds: expected, named, never softened + +A check that goes red for a filed finding stays red — softening it would hide the next real +break behind the same shape. What it gets instead is a NAME in its failure message, so a reader +scanning a gate report recognizes it in one line instead of chasing it as fresh breakage. This is +how the W5 steer red is handled, and it now applies to one more: + +- **`matrix_h1_bad_harness.py`, the `null_kind` case — finding SF2.** A cleared harness + (`{"kind": null}`) is not rejected: it silently defaults to `pi_core`, so on any config whose + model spelling suits Pi the turn runs and the cell correctly fails. Filed for the next release, + not fixed in v0.114.4. The failure message says so. When SF2 is fixed the case turns green on + its own and the `known_finding` key stops appearing — that is the signal to delete the note. + A wrong-type or unknown-string harness that runs is a DIFFERENT, unfiled defect and is + deliberately not covered by the name. + ## The checklist for the next QA run 1. `docker ps` — is anything restarting? If yes, wait. diff --git a/.agents/skills/agent-release-gate/resources/check_secrets_teardown.py b/.agents/skills/agent-release-gate/resources/check_secrets_teardown.py new file mode 100644 index 0000000000..9fd886c753 --- /dev/null +++ b/.agents/skills/agent-release-gate/resources/check_secrets_teardown.py @@ -0,0 +1,530 @@ +# /// script +# requires-python = ">=3.10" +# dependencies = ["httpx>=0.27"] +# /// +"""TIER: mechanical (no model discovery). One short Daytona journey, then an inventory check +against the Daytona API. Never cite for a model behaviour claim. + +Daytona Secrets must not outlive the run that created them. A Daytona run stores the real model +key as a Daytona Secret and gives the sandbox only a `dtn_secret_` placeholder. Teardown +deletes the Secret. When teardown misses one, the key stays live in the Daytona organization with +no sandbox left to use it: a credential leak that no product surface shows, that grows silently +with every run, and that nothing else in the gate would ever notice. + +WHY STANDALONE AND NOT FOLDED INTO matrix_w1_daytona.py. Three reasons, in order of weight. The +teardown assertion needs a SECOND credential the rest of the gate does not use (a Daytona API key +with Secrets read access), so folding it in would give an existing green cell a new way to SKIP +for a reason unrelated to what it tests. It needs an eviction step W1 does not have and does not +want. And it needs a before/after inventory around the whole journey, which is a different shape +from W1's single round trip. Keeping it separate costs one small duplicated config helper and +keeps both cells honest about what their result means. + +HOW THE RUN IS TORN DOWN. There is no product route that closes a session, so the cell uses the +product's own teardown trigger: a second turn on the SAME session with a CHANGED configuration. +The runner reads that as a config mismatch and evicts the sandbox for a cold rebuild, which runs +the Secret deletion path. This is the same mechanism `matrix_l1_lifecycle_routes.py` drives. + +RUN IT ALONE. The check identifies this run's Secrets as the `agenta_*` names that appeared +between the opening and closing inventories. A concurrent Daytona run against the same Daytona +organization will therefore show up as a leftover. The names are printed so an operator can tell +the two apart, but the honest way to read a FAIL is: re-run it alone before believing it. + + PASS no Secret created during the journey is still present after the eviction settles. + FAIL at least one remains; its NAME is printed. + SKIP no Daytona API key in the environment, the Secrets API did not answer, the inventory + could not be enumerated completely, the project vault has no usable Daytona connection, + or the provider key is out of credit. Printed with the exact reason. + +AN EXHAUSTED KEY IS NOT A DEFECT. A journey that dies on a spent provider key never creates a +Secret, so there is nothing to assert teardown on. It SKIPs with "environment: provider key out +of credit" rather than failing, because a FAIL here would point at the teardown path when nothing +about it was exercised. Recognition is narrow (`out_of_credit` in `qa_matrix_lib`): the runner's +own credits copy and the provider's billing refusal, never a bare 401 and never a rate limit. + +THE SECRETS API, AND WHY THE OBVIOUS SPELLING IS WRONG. Verified against +`@daytona/api-client@0.198.0` inside the running runner, and live by status code: + + /secret/paginated the listing. Cursor-paginated, 100 per response. + /secret/{secretId} one Secret; a 404 means it is gone. This is what the settle loop polls. + /secret the unpaginated listing. DEPRECATED, and per the client's own docs it + "fails for organizations with more than 1500 secrets" -- unusable here. + /secrets does not exist. It 404s with "Cannot GET", while /secret and + /secret/paginated return 403 for an under-scoped key. That difference is + how the correct path was confirmed without list access. + +A `page` parameter is SILENTLY IGNORED by the listing: the same 100 ids come back for every +"page". Only `cursor` advances. With ~3510 secrets in the organization, an unpaginated or +page-based inventory makes the before/after difference noise in both directions -- it would +invent leftovers and hide real ones at the same time. So the enumeration follows `nextCursor` to +exhaustion, refuses to loop on a repeating cursor, and is bounded by both a page ceiling and a +wall-clock budget. Hitting either bound is a SKIP: a partial inventory cannot produce a verdict. + +NEVER PRINTS A VALUE. The cell reports Secret NAMES, ids and counts only. A name is a random +handle (`agenta__`, from `generatedName` in +`services/runner/src/engines/sandbox_agent/daytona-secrets.ts`); the value is the model key and +never enters the output, the result JSON, or an exception message. The listing payload does not +carry a value field at all, so a listing cannot leak one even by accident. + + uv run check_secrets_teardown.py + uv run check_secrets_teardown.py --settle 90 +""" + +import argparse +import json +import os +import pathlib +import re +import sys +import time +import uuid +from urllib.parse import urlparse + +import httpx + +sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent)) +from qa_matrix_lib import ( # noqa: E402 + archive, + create_workflow, + invoke, + out_of_credit, + refs, + seed_and_baseline, + user_msg, +) + +BASELINE = "Be terse. Answer in one word." + +#: The shape `generatedName` mints: `agenta_` + 18 random bytes as hex + `_` + the plan ordinal. +RUN_SECRET_NAME = re.compile(r"^agenta_[0-9a-f]{36}_\d+$") + +#: What the paginated listing returns per response. The server caps it at 100. +PAGE_LIMIT = 100 + +#: Bounds on a full enumeration, so a huge organization cannot make the cell hang. Both are +#: deliberately generous: a ~3510-secret org needs ~36 pages. Hitting either bound is a SKIP, +#: because a partial inventory produces a set difference that is noise, not evidence. +MAX_PAGES = 200 +MAX_ENUMERATION_SECONDS = 120.0 + +#: How long to wait between settle polls, capped by whatever remains of the caller's budget. +SETTLE_POLL_SECONDS = 5.0 + +#: A floor for the FIRST probe only. That read measures what is already true rather than waiting +#: for a deletion, so `--settle 0` still gets one honest look instead of a zero-second timeout. +INITIAL_PROBE_SECONDS = 10.0 + +#: The vault resolver's OWN diagnostics, as whole phrases. Deliberately not the bare nouns +#: `credential` and `connection`: those appear in transport failures and in +#: `credential_delivery_failed` itself, so matching them turned real defects into green SKIPs. +MISSING_CREDENTIAL_PHRASES = ( + "not found for provider", + "no connections for provider", + "no connections", + "multiple connections for provider", + "multiple connections", + "requires an effective https endpoint", + "no usable credential", +) + +#: A real failure, whatever else the message happens to mention. Checked first, and never a SKIP. +TRANSPORT_FAILURE_MARKERS = ( + "econnreset", + "econnrefused", + "connection reset", + "connection refused", + "connection error", + "enotfound", + "eai_again", + "timed out", + "timeout", + "502", + "503", + "504", + "credential_delivery_failed", + "credentials from reaching the model", +) + + +class SkipCheck(Exception): + """Raised for a condition that leaves the invariant untested, never for a real failure.""" + + +def environment_cause(error_text: str) -> str | None: + """Why this ONE error frame is an environment condition, or None when it is a real failure. + + SKIP means "the product was never tested here", so the bar for it is evidence that the run + could not start, not merely that some credential word appears. The previous version matched + `credential` and `connection` anywhere in the joined error text, which swept in exactly the + failures this check exists to catch: a connection reset, an upstream connection error, or a + `credential_delivery_failed` timeout all contain one of those words and would have become a + green SKIP. + + So a transport or service failure is checked FIRST and always wins: those are real, and a + vault phrase appearing beside one does not excuse it. What remains is matched against the + vault resolver's own diagnostics, which are specific sentences rather than bare nouns. + """ + low = error_text.lower() + if any(marker in low for marker in TRANSPORT_FAILURE_MARKERS): + return None + spent = out_of_credit(error_text) + if spent: + return spent + if any(phrase in low for phrase in MISSING_CREDENTIAL_PHRASES): + return "missing or ambiguous Daytona vault credential" + return None + + +def daytona_key() -> str: + key = os.environ.get("DAYTONA_API_KEY") or os.environ.get( + "AGENTA_RUNNER_DAYTONA_API_KEY" + ) + if not key: + raise SkipCheck( + "no Daytona API key in the environment (DAYTONA_API_KEY or " + "AGENTA_RUNNER_DAYTONA_API_KEY). The Secrets inventory cannot be read without one, " + "so teardown is unverified for this run." + ) + return key + + +def daytona_api_url() -> str: + """The Daytona API base, refused unless it is HTTPS. + + `_get` sends the API key as a bearer token, so an `http://` base would put a live credential + on the wire in cleartext. Both env vars are operator-set and a typo is the likely cause, so + this refuses loudly rather than downgrading silently. A SKIP naming the scheme is a far better + outcome than a leaked key and a green check. + """ + raw = ( + os.environ.get("DAYTONA_API_URL") + or os.environ.get("AGENTA_RUNNER_DAYTONA_API_URL") + or "https://app.daytona.io/api" + ).rstrip("/") + if urlparse(raw).scheme.lower() != "https": + raise SkipCheck( + f"the Daytona API base is not HTTPS ({raw!r}); refusing to send the API key over " + "a cleartext connection. Fix DAYTONA_API_URL or AGENTA_RUNNER_DAYTONA_API_URL." + ) + return raw + + +def _get( + path: str, params: dict | None = None, timeout: float = 30.0 +) -> httpx.Response: + url = f"{daytona_api_url()}{path}" + try: + return httpx.get( + url, + params=params, + headers={"Authorization": f"Bearer {daytona_key()}"}, + timeout=timeout, + ) + except httpx.HTTPError as e: + raise SkipCheck( + f"the Daytona Secrets API is not reachable at {url}: {e}" + ) from e + + +def list_secret_ids_by_name(fetch=None) -> dict[str, str]: + """Every Secret in the organization as `{name: id}`. Names and ids only; never a value. + + Enumerates `GET /secret/paginated` by CURSOR to exhaustion. Three traps are load-bearing here, + all of them found live against a ~3510-secret organization: + + 1. `GET /secret` (the unpaginated route) is DEPRECATED and, per the api-client's own docs, + "fails for organizations with more than 1500 secrets". It cannot be used here at all. + 2. The listing is cursor-paginated at 100 per response. A `page` parameter is SILENTLY + IGNORED: the same 100 ids come back for every "page", so anything built on `page` produces + a set difference that is pure noise in both directions. Only `cursor` advances. + 3. `/secrets` (plural) does not exist and 404s. The real paths are `/secret`, + `/secret/paginated` and `/secret/{secretId}`. + + `fetch` is a seam for tests: a callable taking `(cursor)` and returning the decoded page. + """ + fetcher = fetch or _fetch_page + by_name: dict[str, str] = {} + cursor: str | None = None + seen_cursors: set[str] = set() + deadline = time.time() + MAX_ENUMERATION_SECONDS + + for page in range(MAX_PAGES): + if time.time() > deadline: + raise SkipCheck( + f"enumerating Daytona Secrets exceeded {MAX_ENUMERATION_SECONDS}s after " + f"{page} page(s) and {len(by_name)} secret(s). An incomplete inventory cannot " + "produce a verdict, so this is a SKIP rather than a guess." + ) + body = fetcher(cursor) + # Validate the SHAPE before touching it. A malformed page must be a SKIP naming what came + # back, never an AttributeError or TypeError escaping mid-walk: the cell would then abort + # with a stack trace instead of a verdict, which reads as a broken gate rather than an + # unreadable inventory. `fetch` is a test seam, so this also holds for injected pages. + if not isinstance(body, dict): + raise SkipCheck( + f"unexpected Daytona Secrets payload shape: body is {type(body).__name__}" + ) + items = body.get("items") + if not isinstance(items, list): + raise SkipCheck( + f"unexpected Daytona Secrets payload shape: items is {type(items).__name__}" + ) + for row in items: + if isinstance(row, dict) and row.get("name") and row.get("id"): + by_name[str(row["name"])] = str(row["id"]) + + next_cursor = body.get("nextCursor") + if next_cursor is not None and not isinstance(next_cursor, str): + # A list or dict here would be unhashable or unusable, and `cursor in seen_cursors` + # would raise instead of skipping. The walk cannot continue from a cursor it cannot + # send, and a partial inventory produces no verdict. + raise SkipCheck( + "unexpected Daytona Secrets payload shape: nextCursor is " + f"{type(next_cursor).__name__}" + ) + cursor = next_cursor or None + if cursor is None: + return by_name + # A cursor that repeats is not advancing. Left unchecked that is an infinite loop, and it + # is exactly the shape the ignored `page` parameter has. + if cursor in seen_cursors: + raise SkipCheck( + f"the Daytona Secrets cursor stopped advancing after {len(by_name)} secret(s); " + "the inventory is incomplete, so no verdict is possible" + ) + seen_cursors.add(cursor) + + raise SkipCheck( + f"enumerating Daytona Secrets hit the {MAX_PAGES}-page ceiling " + f"({len(by_name)} secret(s) seen) without reaching the end of the cursor. An incomplete " + "inventory cannot produce a verdict." + ) + + +def _fetch_page(cursor: str | None) -> dict: + params: dict[str, object] = {"limit": PAGE_LIMIT} + if cursor: + params["cursor"] = cursor + r = _get("/secret/paginated", params) + if r.status_code != 200: + raise SkipCheck( + f"GET /secret/paginated answered HTTP {r.status_code}: {r.text[:200]}" + ) + try: + body = r.json() + except ValueError as e: + raise SkipCheck("GET /secret/paginated returned a non-JSON body") from e + if not isinstance(body, dict): + raise SkipCheck( + f"unexpected Daytona Secrets payload shape: {type(body).__name__}" + ) + return body + + +def secret_exists(secret_id: str, timeout: float = 30.0) -> bool: + """Is this Secret still present? `GET /secret/{secretId}`; a 404 means it is gone. + + Polling by id keeps the settle loop O(secrets this run created) instead of re-enumerating a + ~3510-secret organization every few seconds. + """ + r = _get(f"/secret/{secret_id}", timeout=timeout) + if r.status_code == 200: + return True + if r.status_code == 404: + return False + raise SkipCheck( + f"GET /secret/{{id}} answered HTTP {r.status_code}, so whether the Secret survived " + f"teardown is unknown: {r.text[:200]}" + ) + + +def daytona_agent_config(instructions: str) -> dict: + return { + "instructions": {"agents_md": instructions}, + "llm": { + "model": "haiku", + "provider": "anthropic", + "connection": {"mode": "agenta", "slug": None}, + "extras": {}, + }, + "tools": [], + "mcps": [], + "skills": [], + "harness": {"kind": "claude"}, + "sandbox": {"kind": "daytona"}, + } + + +def secrets_teardown(settle_seconds: int) -> dict: + # Read the inventory BEFORE anything else: a missing key or an unreachable API is a SKIP, and + # a SKIP must not leave a workflow behind. + before = set(list_secret_ids_by_name()) + + hexid = uuid.uuid4().hex[:8] + wf, var = create_workflow(hexid, "qa-secteardown") + try: + cfg = daytona_agent_config(BASELINE) + rev_id, _ver = seed_and_baseline(wf, var, cfg, hexid) + references = refs(wf, var, rev_id) + session_id = str(uuid.uuid4()) + + t1 = invoke( + session_id, + [user_msg("Reply with exactly the word READY and nothing else.")], + {"agent": cfg}, + references, + ) + if t1.errors: + # Classify EACH error frame, and SKIP only when every one of them is an environment + # cause. Joining the frames first let a single credit phrase excuse a transport or + # delivery failure sitting beside it, turning a real teardown miss into a green SKIP. + # A mixed window is not an environment window. + unexplained = [e for e in t1.errors if environment_cause(e) is None] + if not unexplained: + reason = environment_cause(t1.errors[0]) or "environment condition" + raise SkipCheck(f"{reason}: {t1.errors[0][:200]}") + return { + "status": "FAIL", + "why": ( + f"the journey never ran, and {len(unexplained)} of {len(t1.errors)} error " + f"frame(s) name no environment cause: {unexplained[0][:200]!r}" + ), + "session_id": session_id, + "workflow_id": wf, + } + + during = list_secret_ids_by_name() + created = sorted(n for n in (set(during) - before) if RUN_SECRET_NAME.match(n)) + created_ids = {n: during[n] for n in created} + if not created: + raise SkipCheck( + "the journey created no Secret with the runner's generated-name shape, so there " + "is nothing to assert teardown on. Either the run reused a warm sandbox, or this " + "deployment does not deliver credentials as Daytona Secrets." + ) + + # Force the eviction. A changed configuration on the same session is a config mismatch, + # which the runner answers by tearing the sandbox down and rebuilding cold. That teardown + # is what deletes the Secrets. + evict_cfg = daytona_agent_config(BASELINE + " Always end with a full stop.") + invoke( + session_id, + [user_msg("Reply with exactly the word AGAIN and nothing else.")], + {"agent": evict_cfg}, + references, + ) + + # Deletion is asynchronous. Poll instead of sleeping once, so a fast teardown finishes + # fast and a slow one still gets its full budget. Poll each created Secret BY ID rather + # than re-enumerating the organization: a full enumeration is ~36 requests here, and + # running that every few seconds would cost more than the whole rest of the cell. + # Check ONCE immediately, then poll within what is left of the budget. The earlier + # version slept a flat 5s before its first look, so `--settle 1` reported PASS on a + # deletion that took five times its budget, and `--settle 0` never looked at all — the + # deadline was decorative. `monotonic` because a wall-clock step would corrupt it. + # + # Each probe is bounded by the REMAINING budget, not only the gaps between them. A probe + # carries its own request timeout, so a slow one could otherwise complete long after the + # deadline, come back 404, and let the loop report a within-budget deletion it never + # observed within budget. The first read gets a floor: it measures what is already true + # rather than waiting for anything, so `--settle 0` still gets one honest look. + deadline = time.monotonic() + settle_seconds + + def remaining() -> float: + return deadline - time.monotonic() + + def still_present(budget: float) -> list: + return sorted( + n for n in created if secret_exists(created_ids[n], timeout=budget) + ) + + # TIMESTAMP THE OBSERVATION, not just the polling. The first read carries a floor so it + # can complete at all, which means it may itself finish after the deadline on a short + # budget -- and an absence FIRST SEEN after the deadline is not evidence of an in-budget + # deletion, however true the absence is. Recording when the absence was observed is what + # keeps the PASS claim ("deleted within Ns") honest; without it the floor quietly reopened + # the very deadline hole the polling fix closed. + leftover = still_present(max(remaining(), INITIAL_PROBE_SECONDS)) + observed_at = time.monotonic() + while leftover and remaining() > 0: + time.sleep(min(SETTLE_POLL_SECONDS, remaining())) + budget = remaining() + if budget <= 0: + break + leftover = still_present(budget) + observed_at = time.monotonic() + + if leftover: + return { + "status": "FAIL", + "why": ( + f"{len(leftover)} of {len(created)} Secret(s) created by this run are still " + f"listed {settle_seconds}s after the eviction. Re-run this check alone " + "before believing it: a concurrent Daytona run shows the same way." + ), + "leftover_secret_names": leftover, + "created_secret_names": created, + "session_id": session_id, + "workflow_id": wf, + } + if observed_at > deadline: + # Deleted, but NOT proven within the requested budget. Deliberately not a FAIL: no + # Secret outlived its run, so the invariant this cell guards was never violated, and + # reporting a leak that did not happen is how a standing check earns a reputation for + # crying wolf. Deliberately not a PASS either: the budgeted claim is unproven, and the + # PASS line would state a number nobody measured. A SKIP is the cell's honest verdict + # for "the thing you asked was not tested", and the gate counts every SKIP as a + # failure to explain, so it stays visible rather than passing quietly. + raise SkipCheck( + f"all {len(created)} Secret(s) created by this run are gone, but the absence was " + f"first observed {observed_at - deadline:.1f}s AFTER the {settle_seconds}s settle " + "budget, so deletion within that budget is unproven. No Secret outlived its run. " + "Re-run with a larger --settle to turn this into a verdict." + ) + return { + "status": "PASS", + "why": ( + f"all {len(created)} Secret(s) created by this run were deleted within " + f"{settle_seconds}s of the eviction" + ), + "created_secret_names": created, + "session_id": session_id, + "workflow_id": wf, + } + finally: + archive(wf) + + +def main() -> int: + p = argparse.ArgumentParser( + description="Fail when a Daytona Secret outlives the run that created it." + ) + p.add_argument( + "--settle", + type=int, + default=60, + help="seconds to wait for asynchronous Secret deletion after the eviction (default: 60)", + ) + args = p.parse_args() + + try: + r = secrets_teardown(args.settle) + except SkipCheck as e: + r = {"status": "SKIP", "why": str(e)} + except Exception as e: # noqa: BLE001 -- classify infra faults, never crash the run + msg = str(e) + reason = environment_cause(msg) + if reason: + r = {"status": "SKIP", "why": f"{reason}: {msg}"} + else: + r = { + "status": "FAIL", + "why": f"unhandled exception: {type(e).__name__}: {msg}", + } + + print("\n=== SECRETS-TEARDOWN RESULT ===") + print(json.dumps(r, indent=2, default=str)) + return 0 if r["status"] in ("PASS", "SKIP") else 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.agents/skills/agent-release-gate/resources/matrix_c5_first_call_race.py b/.agents/skills/agent-release-gate/resources/matrix_c5_first_call_race.py new file mode 100644 index 0000000000..587ecbf0c0 --- /dev/null +++ b/.agents/skills/agent-release-gate/resources/matrix_c5_first_call_race.py @@ -0,0 +1,577 @@ +# /// script +# requires-python = ">=3.10" +# dependencies = ["httpx>=0.27"] +# /// +"""TIER: mechanical (no model discovery). The prompt asks for one short reply, so the turn makes +ONE model call and makes it as early as a cold sandbox can. Never cite this cell for a model +behaviour claim; it tests credential DELIVERY, not the agent. + +C5: the first-call placeholder race. On a Daytona run the real model key never enters the sandbox. +It is stored as a Daytona Secret, and the sandbox holds a `dtn_secret_` placeholder that +Daytona substitutes into egress to the key's exact host. That substitution propagates +ASYNCHRONOUSLY with no confirmation signal (measured 10-24s after Secret creation). When a fresh +sandbox's FIRST outbound model call beats the propagation, the provider receives the raw +placeholder and refuses it with a 401. + +WHAT THIS CELL PINS. Not that the race never happens -- it is a vendor-side timing property and it +will happen again. What it pins is that the race is never REPORTED AS THE USER'S FAULT. A +placeholder 401 is `credential_delivery_failed` with retry copy. The failure this cell exists to +catch is the production incident of 2026-08-30: a free-credits user hit the race and the product +told them to add their own OpenAI key, which was wrong three ways (their key was fine, adding one +would not have helped, and the run was retryable). + + PASS the turn succeeds, OR the run fails with code `credential_delivery_failed`. + FAIL the run advises adding a key (a `starter_credits_*` code, or "add ... key" wording) while + the underlying refusal carries the placeholder signature (`Received=dtn_`/`dtn_secret_`). + FAIL the run advises adding a key over ANY credential refusal, echo or no echo. See below. + FAIL any other error, or a stored ledger row that never appeared. + SKIP the project vault has no usable Daytona connection, or the provider key is out of + credit (printed with the exact reason). + +AN EXHAUSTED KEY IS NOT A DEFECT. A run that dies on a spent provider key never reached the race, +so it says nothing about the product. It SKIPs with "environment: provider key out of credit" +rather than failing. Recognition is narrow (`out_of_credit` in `qa_matrix_lib`): it matches the +runner's own credits copy and the provider's billing refusal, never a bare 401 and never a rate +limit. This check runs AFTER the incident condition, which is strictly more specific -- a +placeholder refusal dressed in add-a-key copy stays a FAIL even when that copy names credits. + +COLD START IS THE POINT. The cell mints a brand new workflow and variant per run, so the session +pool key has never been seen and the sandbox is necessarily created cold. A warm reuse would make +the cell green for the wrong reason -- a warm sandbox's Secret was substituted minutes ago. + +WHY THE ASSERTION IS BODY-INDEPENDENT. The first version of this cell only failed when the refusal +body echoed the placeholder, which meant it could not see the majority path. Only the litellm +credits proxy echoes ("Received=dtn_****"); `api.anthropic.com` answers an unsubstituted +placeholder with "Invalid bearer token" and echoes nothing at all, and OpenAI's echo is masked +past the literal `dtn_secret_`. So on a direct provider -- where BYO-key cloud users live -- the +echo test is blind, and F6 shipped a user-blaming 401 straight through this cell. + +The stronger assertion needs no body evidence: this cell's sandbox is necessarily fresh, so its +model key was delivered as a Daytona Secret seconds ago, and a credential refusal on such a run +must NEVER advise adding a key. With PR #6408 the honest classification is +`credential_delivery_failed` on the first occurrence in a session (the second falls through to +add-a-key advice by design, which is why this cell uses a new session per run). Against a +deployment that predates #6408 the assertion fails by construction; pass `--pre-6408` to report +that as a SKIP naming the known gap rather than as an unexplained failure. + +STORED-ROW ASSERTION. The turns table has NO error column (verified against +`api/oss/src/dbs/postgres/sessions/turns/dbas.py`: session_id, turn_id, stream_id, turn_index, +harness_kind, agent_session_id, sandbox_id, references, trace_id, span_id, start_time, end_time). +So the stored assertion is the ledger ROW -- it must exist, and it must name the sandbox the turn +ran on. An empty ledger is MISSING EVIDENCE and fails; it is never read as stability. The coded +error surface is the `data-agent-error` frame, whose `code` field is the runner's stable class. + +PROXY LOG, AND THE CONTAINER IT IS READ FROM. When the deployment is local and a credits proxy +belongs to it, the cell counts `Received=dtn_` lines in that proxy since the cell started and +reports the count. The count is diagnostic, never a verdict: a race the runner classified +correctly is a PASS even when the proxy logged the refusal. + +The container is never guessed by name alone. An earlier version took the first `docker ps` name +containing "litellm", which on a shared box matched `starter-litellm-proxy` -- a DIFFERENT +project's container, while the target stack had no proxy at all. Reading a foreign container's log +is worse than reading none: it invents evidence about a deployment that was never under test. So +resolution is, in order: an explicit `--proxy-container`; otherwise a container whose compose +project label equals the TARGET stack's project (from `--compose-project`, or derived by finding +which container publishes the port in `AGENTA_BASE`). When nothing matches, the cell prints +"no credits proxy in this deployment; count not applicable" and carries on. + + uv run matrix_c5_first_call_race.py + uv run matrix_c5_first_call_race.py --proxy-container agenta-ee-dev-rel1144-litellm-proxy-1 + uv run matrix_c5_first_call_race.py --compose-project agenta-ee-dev-rel1144 +""" + +import argparse +import json +import pathlib +import re +import subprocess +import sys +import time +import uuid +from urllib.parse import urlparse + +sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent)) +from qa_matrix_lib import ( # noqa: E402 + BASE, + archive, + create_workflow, + invoke, + out_of_credit, + refs, + seed_and_baseline, + turn_ledger, + user_msg, +) + +BASELINE = "Be terse. Answer in one word." + +#: The runner's own class for a placeholder-shaped refusal. Source of truth: +#: `services/runner/src/engines/sandbox_agent/errors.ts`. +CREDENTIAL_DELIVERY_FAILED = "credential_delivery_failed" + +#: Codes whose user-facing copy tells the reader to add their own provider key. Correct when the +#: credits really are gone; a false accusation when the refusal was a placeholder. +ADD_A_KEY_CODES = ( + "starter_credits_exhausted", + "starter_credits_program_paused", + "starter_credits_unavailable", +) + +#: The placeholder signature, mirroring `PLACEHOLDER_CREDENTIAL` in the runner's errors.ts. The +#: first alternative is LiteLLM refusing a non-`sk-` bearer; the second is any provider echoing +#: the placeholder itself. +PLACEHOLDER_SIGNATURE = re.compile( + r"virtual key expected.*received=dtn_|dtn_secret_", re.I +) + +#: Prose that advises adding a key, for a runner that reports the advice without one of the coded +#: classes above. Deliberately narrow: it must not match the honest retry copy. +ADD_A_KEY_WORDING = re.compile( + r"add (?:your own |the project's |a )?[\w .'-]*\bkey\b", re.I +) + +#: A refusal of the credential itself, whatever the provider calls it. Mirrors `AUTH_REFUSAL` in +#: the runner's errors.ts, plus Anthropic's own wording. This is what makes the assertion below +#: body-INDEPENDENT: a direct provider names no placeholder, so the only thing to key on is that +#: the run was refused for its credential at all. +AUTH_CLASS = re.compile( + r"(? bool: + """Is this failure the vault having no usable connection, as opposed to anything going wrong? + + SKIP claims the product was never tested, so it needs evidence the run could not start. A + transport or service failure is checked first and always wins: it is real, and a vault phrase + appearing beside one does not excuse it. + """ + low = error_text.lower() + if any(marker in low for marker in TRANSPORT_FAILURE_MARKERS): + return False + return any(phrase in low for phrase in MISSING_CREDENTIAL_PHRASES) + + +def key_blame_verdict( + codes: list, error_text: str, placeholder_seen: bool, pre_6408: bool = False +) -> dict | None: + """The verdict when this run blamed the USER'S KEY, or None when it did not. + + Two conditions, in order of evidence strength. Both are failures; they differ only in what + they can prove, and the message says which. + + 1. Add-a-key advice over a body that echoes the placeholder. The original signature: the + refusal itself proves the key never reached the provider. + 2. Add-a-key advice over ANY credential refusal on this cell's necessarily-fresh Daytona + sandbox, echo or no echo. This is the assertion the first version could not make, and it + is the one that matters on a direct provider: `api.anthropic.com` answers an unsubstituted + placeholder with "Invalid bearer token" and echoes nothing, so condition 1 is blind + exactly where BYO-key cloud users live. See the F6 investigation and PR #6408. + + A run on a pre-#6408 runner fails condition 2 by construction, because the honest + classification did not exist yet. `pre_6408` turns that one case into a SKIP naming the + reason, so an older deployment reports a known gap instead of an unexplained failure. It + never softens condition 1, which every shipped version has been expected to catch. + """ + advises_key = any(c in ADD_A_KEY_CODES for c in codes) or bool( + ADD_A_KEY_WORDING.search(error_text) + ) + if not advises_key: + return None + if placeholder_seen: + return { + "status": "FAIL", + "why": ( + "a placeholder refusal was reported as the user's key problem: " + f"codes={codes}, error={error_text[:400]!r}" + ), + } + if not AUTH_CLASS.search(error_text): + return None + if pre_6408: + return { + "status": "SKIP", + "why": ( + "this deployment predates PR #6408, so a credential refusal on a fresh Daytona " + "sandbox still carries add-a-key copy by construction. That is the known F6 gap, " + "not a new defect, and this cell cannot test the invariant here: " + f"codes={codes}, error={error_text[:300]!r}" + ), + } + return { + "status": "FAIL", + "why": ( + "a credential refusal on a FRESH Daytona sandbox was reported as the user's key " + "problem. The body echoes no placeholder, but a direct provider never echoes one, so " + "that proves nothing: this run's key was delivered as a Daytona Secret moments " + "earlier and the honest classification is " + f"{CREDENTIAL_DELIVERY_FAILED}. codes={codes}, error={error_text[:400]!r}" + ), + } + + +def daytona_agent_config() -> dict: + return { + "instructions": {"agents_md": BASELINE}, + "llm": { + "model": "haiku", + "provider": "anthropic", + "connection": {"mode": "agenta", "slug": None}, + "extras": {}, + }, + "tools": [], + "mcps": [], + "skills": [], + "harness": {"kind": "claude"}, + "sandbox": {"kind": "daytona"}, + } + + +def _base_is_local() -> bool: + return any(h in BASE for h in ("localhost", "127.0.0.1", "0.0.0.0")) + + +def _docker(args: list[str], timeout: float = 20.0) -> str | None: + """Run a docker command, or return None when docker cannot answer.""" + try: + out = subprocess.run( + ["docker", *args], capture_output=True, text=True, timeout=timeout + ) + except (OSError, subprocess.SubprocessError): + return None + return out.stdout if out.returncode == 0 else None + + +def _running(container: str) -> bool: + out = _docker(["ps", "--format", "{{.Names}}"]) + return out is not None and container in [n.strip() for n in out.splitlines()] + + +def target_compose_project() -> str | None: + """The compose project of the stack `AGENTA_BASE` points at, found via its published port. + + Derived rather than assumed: the container that publishes the port in `AGENTA_BASE` IS the + target deployment, so its `com.docker.compose.project` label is the only project whose + containers this cell may read. + """ + port = urlparse(BASE).port or (443 if BASE.startswith("https") else 80) + out = _docker(["ps", "--format", "{{.Names}}\t{{.Ports}}"]) + if out is None: + return None + for line in out.splitlines(): + name, _, ports = line.partition("\t") + if f":{port}->" not in ports: + continue + label = _docker( + [ + "inspect", + "-f", + '{{index .Config.Labels "com.docker.compose.project"}}', + name.strip(), + ] + ) + return label.strip() if label and label.strip() else None + return None + + +def resolve_proxy_container( + explicit: str | None, compose_project: str | None +) -> tuple[str | None, str]: + """The credits proxy belonging to the TARGET stack, or `(None, why)`. + + Never falls back to a name match across the whole box. On a shared host that is how a foreign + project's proxy gets read, which invents evidence about a deployment that was never tested. + """ + if explicit: + if not _running(explicit): + return None, f"proxy container {explicit!r} is not running" + return explicit, f"using --proxy-container {explicit}" + if not _base_is_local(): + return None, f"proxy log not reachable (AGENTA_BASE={BASE} is not local)" + + project = compose_project or target_compose_project() + if not project: + return None, ( + "cannot determine the target stack's compose project, so no container may be read; " + "pass --compose-project or --proxy-container" + ) + out = _docker( + [ + "ps", + "--filter", + f"label=com.docker.compose.project={project}", + "--format", + "{{.Names}}", + ] + ) + if out is None: + return None, "proxy log not reachable (docker is not answering)" + hits = [n.strip() for n in out.splitlines() if "litellm" in n.strip().lower()] + if not hits: + return None, ( + f"no credits proxy in this deployment; count not applicable " + f"(compose project {project} has no litellm container)" + ) + return hits[0], f"reading {hits[0]} (compose project {project})" + + +def count_proxy_placeholder_refusals( + since: str, explicit: str | None = None, compose_project: str | None = None +) -> tuple[int | None, str]: + """Count `Received=dtn_` lines in the target stack's proxy log since `since`. + + Returns `(count, why)`. A `None` count means the count is not applicable or the log was not + reachable. Both are reported and neither is ever failed on. + """ + container, why = resolve_proxy_container(explicit, compose_project) + if container is None: + return None, why + try: + out = subprocess.run( + ["docker", "logs", container, "--since", since], + capture_output=True, + text=True, + timeout=30, + ) + except (OSError, subprocess.SubprocessError) as e: + return None, f"proxy log not reachable (docker logs {container} failed: {e})" + lines = (out.stdout + out.stderr).splitlines() + hits = [ln for ln in lines if "Received=dtn_" in ln] + return ( + len(hits), + f"{len(hits)} `Received=dtn_` line(s) in {container} since {since}", + ) + + +def agent_error_frames(turn) -> list[dict]: + """Every coded runner error the turn streamed, as `{"code", "errorText"}` payloads.""" + return [ + f.get("data") or {} + for f in turn.raw_frames + if f.get("type") == "data-agent-error" + ] + + +def c5_first_call_race( + proxy_container: str | None = None, + compose_project: str | None = None, + pre_6408: bool = False, +) -> dict: + hexid = uuid.uuid4().hex[:8] + # A brand new workflow is a pool key never seen before, so the sandbox is created cold and the + # first model call is genuinely a first call. This is the whole premise of the cell. + wf, var = create_workflow(hexid, "qa-c5race") + started_at = time.strftime("%Y-%m-%dT%H:%M:%S", time.gmtime()) + try: + cfg = daytona_agent_config() + rev_id, _ver = seed_and_baseline(wf, var, cfg, hexid) + references = refs(wf, var, rev_id) + session_id = str(uuid.uuid4()) + + # One short prompt, no tools: the turn's first act is the model call, so it lands as early + # after sandbox create as the product allows. + turn = invoke( + session_id, + [user_msg("Reply with exactly the word READY and nothing else.")], + {"agent": cfg}, + references, + ) + + coded = agent_error_frames(turn) + codes = [c.get("code") for c in coded if c.get("code")] + error_text = " ".join( + [str(c.get("errorText") or "") for c in coded] + list(turn.errors) + ) + placeholder_seen = bool(PLACEHOLDER_SIGNATURE.search(error_text)) + proxy_count, proxy_why = count_proxy_placeholder_refusals( + started_at, proxy_container, compose_project + ) + + # The stored row, read back from the API. Empty is missing evidence, never stability. + time.sleep(1.0) + ledger = turn_ledger(session_id) + stored_sandboxes = sorted( + {row.get("sandbox_id") for row in ledger if row.get("sandbox_id")} + ) + evidence = { + "session_id": session_id, + "workflow_id": wf, + "stored_turn_rows": len(ledger), + "stored_sandbox_ids": stored_sandboxes, + "error_codes": codes, + "placeholder_signature_seen": placeholder_seen, + "proxy_placeholder_refusals": proxy_count, + "proxy_log": proxy_why, + "frames": turn.frames, + } + + # The incident: did this run blame the user's key? Checked before every other reading, + # because a run that blamed the key has already failed regardless of what else is true. + blame = key_blame_verdict(codes, error_text, placeholder_seen, pre_6408) + if blame: + return {**blame, **evidence} + + if CREDENTIAL_DELIVERY_FAILED in codes: + if not stored_sandboxes: + return { + "status": "FAIL", + "why": ( + "classified as credential_delivery_failed, but the stored ledger names " + f"no sandbox (rows={len(ledger)}) -- without a sandbox id there is no " + "evidence the turn ran in the cold Daytona sandbox this cell validates" + ), + **evidence, + } + return { + "status": "PASS", + "why": ( + "the race fired and was classified honestly as " + f"{CREDENTIAL_DELIVERY_FAILED} with retry copy; {proxy_why}" + ), + **evidence, + } + + if turn.errors or codes: + # An exhausted key is an environment condition. It is checked AFTER the incident + # condition above, which is strictly more specific: a placeholder refusal dressed in + # add-a-key copy is a real defect even when the copy names credits. It is checked + # BEFORE the vault-credential markers below, which match a bare substring + # ("credential", "connection") and would otherwise claim a credits failure with a + # less accurate reason. + spent = out_of_credit(error_text, codes) + if spent: + return { + "status": "SKIP", + "why": ( + f"{spent}, so the first-call race was never reached: " + f"{error_text[:300]}" + ), + **evidence, + } + if missing_vault_credential(error_text): + return { + "status": "SKIP", + "why": ( + "missing or ambiguous Daytona vault credential, so the first-call race " + f"was never reached: {error_text[:300]}" + ), + **evidence, + } + return { + "status": "FAIL", + "why": f"turn failed for another reason: codes={codes}, error={error_text[:400]!r}", + **evidence, + } + + if not stored_sandboxes: + return { + "status": "FAIL", + "why": ( + "the turn reported success but the stored ledger names no sandbox " + f"(rows={len(ledger)}) -- a row without a sandbox id is not evidence the " + "turn ran in the cold Daytona sandbox this cell validates" + ), + **evidence, + } + if not turn.reply.strip(): + return { + "status": "FAIL", + "why": "the turn produced neither an error nor any text (a silent turn)", + **evidence, + } + return { + "status": "PASS", + "why": ( + f"first call on a cold Daytona sandbox succeeded, reply={turn.reply.strip()[:40]!r}; " + f"{proxy_why}" + ), + **evidence, + } + except Exception as e: # noqa: BLE001 -- classify infra faults as SKIP, never crash the run + msg = str(e) + if missing_vault_credential(msg): + return { + "status": "SKIP", + "why": f"missing or ambiguous Daytona vault credential: {msg}", + } + return { + "status": "FAIL", + "why": f"unhandled exception: {type(e).__name__}: {msg}", + } + finally: + archive(wf) + + +if __name__ == "__main__": + p = argparse.ArgumentParser( + description="Pin that a first-call placeholder 401 is never reported as the user's fault." + ) + p.add_argument( + "--proxy-container", + default=None, + help="credits-proxy container to read the diagnostic count from. Default: a litellm " + "container in the TARGET stack's compose project, and none otherwise.", + ) + p.add_argument( + "--compose-project", + default=None, + help="compose project of the target stack. Default: derived from whichever container " + "publishes the port in AGENTA_BASE.", + ) + p.add_argument( + "--pre-6408", + action="store_true", + help="the target deployment predates PR #6408. Turns the body-independent add-a-key " + "assertion into a SKIP naming the known F6 gap, instead of a failure the reader has to " + "diagnose. Never pass it against a runner that has the fix.", + ) + args = p.parse_args() + r = c5_first_call_race(args.proxy_container, args.compose_project, args.pre_6408) + print("\n=== C5-FIRST-CALL-RACE RESULT ===") + print(json.dumps(r, indent=2, default=str)) + sys.exit(0 if r["status"] in ("PASS", "SKIP") else 1) diff --git a/.agents/skills/agent-release-gate/resources/matrix_h1_bad_harness.py b/.agents/skills/agent-release-gate/resources/matrix_h1_bad_harness.py new file mode 100644 index 0000000000..ccd59c6067 --- /dev/null +++ b/.agents/skills/agent-release-gate/resources/matrix_h1_bad_harness.py @@ -0,0 +1,327 @@ +# /// script +# requires-python = ">=3.10" +# dependencies = ["httpx>=0.27"] +# /// +"""TIER: mechanical (no model discovery). Every case is a malformed configuration driven straight +at the product API. Never cite this cell for a model behaviour claim. + +H1: a malformed harness must fail closed. The harness value selects which coding agent a run +drives (`harness.kind`, one of `pi_core` / `claude` / `codex`). The invariant this cell pins is +narrow and absolute: + + A malformed harness must produce a STRUCTURED REFUSAL at some boundary, and must NEVER run as a + silent Pi turn. + +The dangerous failure is not a crash. It is a DEFAULT: a config whose harness the platform cannot +read, quietly resolved to whichever harness the code falls back to, running a full turn and +storing output the user never asked for. That output looks legitimate afterwards -- the stored +turn row carries a real `harness_kind` -- so nothing downstream can tell it apart from a run the +user configured. The refusal is the only place the truth exists. + +WHICH BOUNDARY REFUSES IS NOT THE POINT, AND THE CELL SAYS SO. Three boundaries can legitimately +own the refusal, and the cell records which one did rather than demanding a particular one: + + commit_api the workflow revision API refuses to persist the malformed config (a clean 4xx). + invoke_http `/services/agent/v0/invoke` refuses the request before streaming. + runner_stream the run starts and the SDK/runner streams a coded `data-agent-error`. + +Per the code as of v0.114.4, the deepest of those is `HarnessKind.coerce` in +`sdks/python/agenta/sdk/agents/dtos.py`, reached from `make_harness` in +`sdks/python/agenta/sdk/agents/adapters/harnesses.py:158`. `coerce` normalizes the value and calls +`cls(normalized)`, which raises `ValueError` for anything that is not a member -- a wrong-type +value included, since `str(value).lower()` of an int is not a member either. A refusal at an outer +boundary is BETTER, not worse, and passes here. + + PASS a boundary refused, the refusal names the harness, and no turn produced output. + FAIL a turn executed and stored output (the defaulted-harness failure this cell exists for). + FAIL nothing refused, or the refusal is unattributable to the harness. + SKIP the run failed for an unrelated infra reason (credentials), so the invariant was never + reached. Printed with the exact reason. + + uv run matrix_h1_bad_harness.py +""" + +import copy +import json +import pathlib +import re +import sys +import time +import uuid + +sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent)) +from qa_matrix_lib import ( # noqa: E402 + agent_config, + archive, + commit_direct, + create_workflow, + invoke, + refs, + seed_and_baseline, + turn_ledger_or_unavailable, + user_msg, +) + +#: A refusal must be attributable to the harness. A generic 500, or a refusal about something +#: else entirely, does not prove the harness was checked. +HARNESS_REFUSAL = re.compile(r"harness|\bkind\b", re.I) + +#: An HTTP failure `invoke` recorded on the turn, as `HTTP : `. +HTTP_STATUS = re.compile(r"^HTTP (\d{3}):") + +MISSING_CREDENTIAL_MARKERS = ( + "connection", + "not found for provider", + "no connections", + "multiple connections", + "credential", + "subscription", + "oauth", +) + +#: Each case is a harness block the platform must not be able to read. `wrong_type` is the one the +#: brief calls for; the other two cover the neighbouring shapes a client can send by accident. +CASES = { + "wrong_type": {"kind": 12345}, + "unknown_string": {"kind": "not_a_harness"}, + "null_kind": {"kind": None}, +} + + +#: The one FAIL shape that is already known, filed, and NOT a fresh regression. +#: +#: A cleared harness (`{"kind": None}`) is not rejected: it silently defaults to `pi_core`, so on +#: any config whose model spelling suits Pi the turn runs and this cell goes red. That is SF2, +#: found by this cell and deliberately filed rather than fixed for this release. +#: +#: The FAIL is NOT softened, because the invariant really is broken -- a malformed harness ran. +#: What the name buys is that the next reader recognizes it in a gate report instead of chasing it +#: as new breakage, which is how W5's standing red is handled. When SF2 is fixed this case turns +#: green on its own and `known_finding` stops appearing; that is the signal to delete this. +SF2_NOTE = ( + "known finding SF2 (cleared harness silently defaults to pi_core), filed for the " + "next release -- expected red, not a fresh regression" +) + + +def known_finding(harness: dict, stored_harnesses: list) -> str | None: + """The note for a FAIL shape that is already filed, or None when the failure is new. + + Narrow on purpose: only a CLEARED harness, and only when nothing contradicts the default. + A wrong-type or unknown-string harness that runs is a different, unfiled defect and must read + as one. + """ + if harness.get("kind") is not None: + return None + # POSITIVE evidence of the pi_core default is required. A stored row whose harness_kind is + # unset proves a turn ran but says nothing about what it ran AS, and SF2 is specifically the + # silent pi_core default. The asymmetry decides this: under-labelling costs an operator one + # investigation of a real FAIL, while over-labelling teaches them to wave past a shape that + # may be a fresh regression. So an unproven default reads as new breakage. + if stored_harnesses != ["pi_core"]: + return None + return SF2_NOTE + + +def bad_config(harness: dict) -> dict: + cfg = copy.deepcopy(agent_config()) + cfg["harness"] = harness + return cfg + + +def coded_errors(turn) -> list[dict]: + return [ + f.get("data") or {} + for f in turn.raw_frames + if f.get("type") == "data-agent-error" + ] + + +def probe(wf: str, var: str, references: dict, name: str, harness: dict) -> dict: + """Drive one malformed harness at both boundaries and report which refused.""" + cfg = bad_config(harness) + boundaries: list[str] = [] + detail: dict = {"case": name, "harness": harness} + + # Boundary 1: can the malformed config even be PERSISTED? A clean 4xx here is a pass, and it + # is the outermost place the invariant can hold. + try: + r = commit_direct( + wf, + var, + {"agent": cfg}, + f"h1 {name}", + f"qa-h1-{name}-{uuid.uuid4().hex[:6]}", + ) + detail["commit_status"] = r.status_code + detail["commit_body"] = r.text[:300] + if 400 <= r.status_code < 500 and HARNESS_REFUSAL.search(r.text): + boundaries.append("commit_api") + except Exception as e: # noqa: BLE001 -- a transport fault here is evidence, not a crash + detail["commit_status"] = None + detail["commit_body"] = f"{type(e).__name__}: {e}" + + # Boundary 2 and 3: run it. The references point at a VALID baseline, so the only malformed + # thing in the request is the live harness -- nothing else can explain a refusal. + session_id = str(uuid.uuid4()) + detail["session_id"] = session_id + turn = invoke( + session_id, + [user_msg("Reply with exactly the word READY and nothing else.")], + {"agent": cfg}, + references, + log=False, + ) + detail["frames"] = turn.frames + + coded = coded_errors(turn) + codes = [c.get("code") for c in coded if c.get("code")] + error_text = " ".join( + [str(c.get("errorText") or "") for c in coded] + list(turn.errors) + ) + detail["error_codes"] = codes + detail["error_text"] = error_text[:400] + + http_refusals = [ + int(m.group(1)) + for m in (HTTP_STATUS.match(e) for e in turn.errors) + if m is not None + ] + if any(400 <= s < 500 for s in http_refusals) and HARNESS_REFUSAL.search( + error_text + ): + boundaries.append("invoke_http") + if codes and HARNESS_REFUSAL.search(error_text): + boundaries.append("runner_stream") + detail["http_status"] = http_refusals or None + + # The failure this cell exists for: a turn that RAN. Read it back from storage, because a + # defaulted harness is only visible after the fact as a stored row with a real harness_kind. + time.sleep(1.0) + ledger, ledger_available = turn_ledger_or_unavailable(session_id) + stored_harnesses = sorted( + {row.get("harness_kind") for row in ledger if row.get("harness_kind")} + ) + produced_output = bool(turn.reply.strip()) or bool(turn.tool_calls) + detail["stored_turn_rows"] = len(ledger) + detail["stored_harness_kinds"] = stored_harnesses + detail["produced_output"] = produced_output + detail["ledger_available"] = ledger_available + detail["reply"] = turn.reply.strip()[:120] + + # EXECUTION EVIDENCE OUTRANKS A LATER REFUSAL. A turn that emitted text or called a tool RAN, + # and a `data-agent-error` arriving afterwards does not undo that: the malformed harness was + # defaulted to something runnable first and complained second. The earlier version required + # `not error_text`, so a streamed error let a genuinely defaulted run pass. A stored + # harness_kind is the same evidence read from the other side, and is checked here too. + # ROW PRESENCE is the evidence, not the truthiness of a field inside it. `stored_harnesses` + # keeps only truthy `harness_kind` values, so a stored row whose kind is missing, null or + # empty collapsed to `[]` and let the probe reach PASS with a turn demonstrably persisted -- + # the same "absence of a field read as absence of the thing" mistake the ledger-availability + # fix addressed one layer up. A PASS here asserts NOTHING was stored, so any row refutes it. + if produced_output or ledger: + detail["status"] = "FAIL" + detail["why"] = ( + f"a malformed harness {harness!r} RAN: produced_output={produced_output}, " + f"stored_turn_rows={len(ledger)}, stored harness_kind={stored_harnesses} -- the " + f"harness was defaulted, not refused (a later refusal does not undo an executed " + f"turn; error={error_text[:200]!r})" + ) + known = known_finding(harness, stored_harnesses) + if known: + detail["known_finding"] = "SF2" + detail["why"] = f"{detail['why']} -- {known}" + return detail + + # A PASS here asserts that NOTHING was stored, so an unanswered ledger query cannot support + # it: that would be the strongest claim drawn from the weakest evidence. `turn_ledger` alone + # cannot tell "no rows" from "no answer", which is why this cell reads availability too. + if not ledger_available: + detail["status"] = "FAIL" + detail["why"] = ( + "the turn ledger did not answer, so there is no evidence the malformed harness " + f"stored nothing; refusing to infer a PASS from a failed query (boundaries={boundaries})" + ) + return detail + + if boundaries: + detail["status"] = "PASS" + detail["refused_by"] = boundaries[0] + detail["why"] = ( + f"refused at {boundaries[0]} (all refusing boundaries: {boundaries})" + ) + return detail + + low = error_text.lower() + if error_text and any(m in low for m in MISSING_CREDENTIAL_MARKERS): + detail["status"] = "SKIP" + detail["why"] = ( + "the run failed on credentials before any harness check, so the invariant was " + f"never reached: {error_text[:200]}" + ) + return detail + + detail["status"] = "FAIL" + detail["why"] = ( + "no boundary refused the malformed harness in a way attributable to it " + f"(commit={detail.get('commit_status')}, http={http_refusals}, codes={codes}, " + f"error={error_text[:200]!r})" + ) + return detail + + +def h1_bad_harness() -> dict: + hexid = uuid.uuid4().hex[:8] + # Created BEFORE the try so the `finally` can never raise UnboundLocalError over the real + # result. A create that fails must surface its own SKIP or FAIL, not a cleanup crash on top + # of it -- the cell's whole contract is that every outcome is explained. + wf: str | None = None + try: + wf, var = create_workflow(hexid, "qa-h1harness") + rev_id, _ver = seed_and_baseline(wf, var, agent_config(), hexid) + references = refs(wf, var, rev_id) + + cases = [probe(wf, var, references, name, h) for name, h in CASES.items()] + statuses = [c["status"] for c in cases] + if "FAIL" in statuses: + status = "FAIL" + why = "; ".join(c["why"] for c in cases if c["status"] == "FAIL") + elif all(s == "SKIP" for s in statuses): + status = "SKIP" + why = "; ".join(c["why"] for c in cases) + else: + status = "PASS" + refused = { + c["case"]: c.get("refused_by") for c in cases if c["status"] == "PASS" + } + skipped = [c["case"] for c in cases if c["status"] == "SKIP"] + why = f"every readable case failed closed: {refused}" + if skipped: + why += f"; skipped (credentials, invariant not reached): {skipped}" + return { + "status": status, + "why": why, + "workflow_id": wf, + "cases": cases, + } + except Exception as e: # noqa: BLE001 -- classify infra faults as SKIP, never crash the run + msg = str(e) + if any(m in msg.lower() for m in MISSING_CREDENTIAL_MARKERS): + return { + "status": "SKIP", + "why": f"missing or ambiguous vault credential: {msg}", + } + return { + "status": "FAIL", + "why": f"unhandled exception: {type(e).__name__}: {msg}", + } + finally: + if wf is not None: + archive(wf) + + +if __name__ == "__main__": + r = h1_bad_harness() + print("\n=== H1-BAD-HARNESS RESULT ===") + print(json.dumps(r, indent=2, default=str)) + sys.exit(0 if r["status"] in ("PASS", "SKIP") else 1) diff --git a/.agents/skills/agent-release-gate/resources/matrix_l1_lifecycle_routes.py b/.agents/skills/agent-release-gate/resources/matrix_l1_lifecycle_routes.py index d96fb72a6a..1ccf54bc93 100644 --- a/.agents/skills/agent-release-gate/resources/matrix_l1_lifecycle_routes.py +++ b/.agents/skills/agent-release-gate/resources/matrix_l1_lifecycle_routes.py @@ -48,11 +48,14 @@ `matrix_l5_live_route_observed.py` for the other half; a green here plus a red there means the runner is keeping a sandbox that is quietly running the old configuration. -THE `model` CASE IS EVIDENCE-ONLY, ON PURPOSE. Changing the model id should move the `model` -facet alone (apply-live, warm), but if a deployment's connection shape is keyed off the model -then the `runtime` facet moves too and the plan escalates -- correctly. Rather than encode a -guess as a blocker, this cell REPORTS the observed route for `model` and blocks only on the four -cases whose routing is unambiguous in the capability table. +THE `model` CASES ARE BLOCKING, AND THERE ARE TWO OF THEM. A same-connection model switch moves +the `model` facet alone (apply-live, warm): an alias switch on claude, and a fully-qualified id +switch on pi_core. The pi_core variant exists because of a caught bug class: the router once +keyed its capability table on the bare literal "pi" while the wire carries "pi_core", so every +playground Pi run fell into the fail-closed all-rebuild row, the live model route never fired, +and a claude-only cell could not see it (#6364). If this case goes red on a facet OTHER than +`model` moving (the shadow line names it), that red is the discovery mechanism working: it names +the next over-eviction to remove, not a reason to demote the case. uv run matrix_l1_lifecycle_routes.py """ @@ -65,6 +68,8 @@ sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent)) from qa_matrix_lib import ( # noqa: E402 LIVE_TOOLS, + PI_CORE_HAIKU_MODEL, + PI_CORE_HARNESS_KIND, agent_config, archive, create_workflow, @@ -117,10 +122,22 @@ ( "model", 1, - False, - "the `model` facet is the other live route (setModel on the running session), but a " - "deployment whose connection shape is keyed off the model moves `runtime` too and " - "correctly escalates -- reported, not blocking", + True, + "the `model` facet is the other live route (setModel on the running session). An " + "alias-to-alias switch on the same self_managed anthropic connection moves no other " + "facet (the connection shape and the resolved modalities are identical), so a rebuild " + "here means the live route is broken", + ), + ( + "model_pi_core", + 1, + True, + "the SAME model-switch assertion on the pi_core harness. This is the standing trap for " + "the wire-spelling class of bug: the lifecycle router once keyed its capability table " + "on the bare literal 'pi' while the wire carries 'pi_core', so every playground Pi run " + "fell into the fail-closed all-rebuild row and every model switch threw the warm " + "sandbox away (fixed in #6364) -- and the claude-only model case above could not see " + "it. A same-provider id switch moves only the `model` facet", ), ] @@ -151,6 +168,10 @@ def mutate(case: str, params: dict) -> dict: if case == "model": agent["llm"]["model"] = "sonnet" return p + if case == "model_pi_core": + # Same provider, same connection, a different fully qualified id: only `model` moves. + agent["llm"]["model"] = "claude-sonnet-5" + return p raise ValueError(f"unknown case {case}") @@ -186,11 +207,30 @@ def l1(): cfg = agent_config(instructions=BASELINE) rev_id, _ = seed_and_baseline(wf, var, cfg, hexid) base_params = {"agent": {**cfg, "tools": LIVE_TOOLS}} + # The pi_core variant of the model case runs the whole session on pi_core with a fully + # qualified id (the harness rejects bare aliases; see the module notes in qa_matrix_lib). + pi_base_params = json.loads(json.dumps(base_params)) + pi_base_params["agent"]["harness"] = {"kind": PI_CORE_HARNESS_KIND} + pi_base_params["agent"]["llm"]["model"] = PI_CORE_HAIKU_MODEL + # Vault-backed auth, not the inherited self_managed default: a self_managed Pi run + # needs a PI_CODING_AGENT_DIR mount, which a gate deployment does not have, and turn 1 + # would then fail before this case tests any lifecycle routing. Matches the Pi + # fixtures in matrix_l5_live_route_observed.py and bench_lib.py. + pi_base_params["agent"]["llm"]["connection"] = { + "mode": "agenta", + "slug": None, + } results = [] blocking_failures = [] for case, expected, blocking, why in CASES: - r = run_case(case, wf, var, rev_id, base_params) + r = run_case( + case, + wf, + var, + rev_id, + pi_base_params if case == "model_pi_core" else base_params, + ) r["expected_sandboxes"] = expected r["blocking"] = blocking r["rationale"] = why diff --git a/.agents/skills/agent-release-gate/resources/qa_matrix_lib.py b/.agents/skills/agent-release-gate/resources/qa_matrix_lib.py index 40a4f14626..834b7b2f52 100644 --- a/.agents/skills/agent-release-gate/resources/qa_matrix_lib.py +++ b/.agents/skills/agent-release-gate/resources/qa_matrix_lib.py @@ -513,7 +513,23 @@ def turn_ledger(session_id: str, limit: int = 20) -> list[dict]: `POST /sessions/turns/`), which makes this a STORED outcome rather than an echo. Returns [] when the ledger is unavailable, which callers must treat as MISSING EVIDENCE and - fail on -- never as evidence of stability.""" + fail on -- never as evidence of stability. A caller that needs to TELL those two apart (a + check whose PASS depends on nothing having been stored) must use `turn_ledger_or_unavailable` + instead; this signature cannot express the difference.""" + rows, _available = turn_ledger_or_unavailable(session_id, limit) + return rows + + +def turn_ledger_or_unavailable( + session_id: str, limit: int = 20 +) -> tuple[list[dict], bool]: + """`(rows, available)` -- the ledger, and whether the query actually answered. + + `turn_ledger` collapses "the query failed" and "the session stored no turn" into the same + empty list. That is safe for a check whose PASS needs rows to EXIST, because both readings + fail. It is unsafe for a check whose PASS needs rows to be ABSENT: a query failure would then + read as proof that nothing ran, which is the strongest possible claim drawn from the weakest + possible evidence. `matrix_h1_bad_harness.py` is exactly that shape.""" r = api_call( "POST", "/sessions/turns/query", @@ -523,8 +539,23 @@ def turn_ledger(session_id: str, limit: int = 20) -> list[dict]: }, ) if r.status_code != 200: - return [] - return r.json().get("turns") or [] + return [], False + # A 200 is not an answer until the payload is the shape the contract promises. `{}` and + # `{"turns": null}` would otherwise read as an answered-EMPTY ledger, which is the one reading + # a caller must never get for free: `matrix_h1_bad_harness.py` turns "answered empty" into a + # PASS asserting nothing was stored. Malformed is unavailable, so that PASS stays unreachable. + try: + body = r.json() + except ValueError: + return [], False + if not isinstance(body, dict): + return [], False + turns = body.get("turns") + if not isinstance(turns, list): + return [], False + if any(not isinstance(row, dict) for row in turns): + return [], False + return turns, True def ledger_ids(session_id: str) -> tuple[list[str], list[str]]: @@ -622,6 +653,58 @@ def run_until_settled( } +# --------------------------------------------------------------------------- +# An exhausted provider key is an ENVIRONMENT condition, not a defect. A cell that renders it as +# FAIL spends a reviewer's attention on a topped-up balance, and worse, it teaches the reader that +# this cell's FAIL is sometimes noise -- which is how a real regression gets waved through later. +# Cells already SKIP on a missing or ambiguous vault credential; a key with no credit left belongs +# in the same class, and reads the same way to a human: nothing about the product was tested. +# +# Recognition is deliberately narrow. It matches the runner's own classified copy, never a bare +# 401 or a rate limit. Source of truth for every string below: +# `services/runner/src/engines/sandbox_agent/errors.ts`. + +#: The runner's coded classes for a credits refusal at the proxy's admission check. +STARTER_CREDIT_CODES = ( + "starter_credits_exhausted", + "starter_credits_program_paused", + "starter_credits_unavailable", +) + +#: Billing-stop prose. The first group is the runner's own user-facing credits copy; the second is +#: the upstream provider's billing refusal, which the runner classifies as `runner_error`, so the +#: code alone cannot catch it. Throttling ("rate limit", "too many requests") is deliberately +#: ABSENT: a throttled run was not out of credit and must stay a FAIL. +_OUT_OF_CREDIT_RE = re.compile( + r"free agenta credits are (?:used up|paused)" + r"|agenta credits are temporarily unavailable" + r"|the model provider account has insufficient credit" + r"|insufficient credit" + r"|no credits remaining" + r"|credit balance is too low" + r"|exceeded your current quota" + r"|insufficient_quota" + r"|budget_exceeded" + r"|budget has been exceeded", + re.I, +) + + +def out_of_credit(error_text: str = "", codes: "list | tuple" = ()) -> str | None: + """The SKIP reason when a run failed ONLY because the provider key has no credit left. + + Returns the explanation to report, or None when the failure is anything else -- in which case + the caller must keep its FAIL. Pass the run's stored/classified error text and any coded + `data-agent-error` classes it carried. + """ + for code in codes or (): + if code in STARTER_CREDIT_CODES: + return f"environment: provider key out of credit ({code})" + if error_text and _OUT_OF_CREDIT_RE.search(error_text): + return "environment: provider key out of credit" + return None + + # --------------------------------------------------------------------------- # The generic invariant: no tool_result with empty output and isError:false may exist for a call # whose runner log says "[commit-auth] refused" (the silent-blank-success class). Added after the diff --git a/.agents/skills/agent-release-gate/resources/sweep_disagree.py b/.agents/skills/agent-release-gate/resources/sweep_disagree.py new file mode 100644 index 0000000000..42832ef2d9 --- /dev/null +++ b/.agents/skills/agent-release-gate/resources/sweep_disagree.py @@ -0,0 +1,307 @@ +# /// script +# requires-python = ">=3.10" +# dependencies = [] +# /// +"""Sweep the runner log for reconciliation DISAGREE lines. Run AFTER a gate session. + +WHAT IT PINS. The runner holds two views of a session's identity: the coordinator's +`configFingerprint` decision, and the reconciliation router's per-facet digests. When those two +views drift apart, a request that should reuse a warm sandbox instead cold-evicts it. That is the +over-eviction family fixed in v0.114.4, and it is invisible from the wire: the turn still +succeeds, it just paid for a rebuild it did not need. The only product of the drift is a log line, +so the only way to catch a regression is to grep for it. + +THE LINE. `logReconcileShadow` in `services/runner/src/lifecycle/reconciliation-router.ts` writes +one line per decision through `defaultLog`, which prefixes `[reconcile] `: + + [reconcile] shadow key= harness= decision=() plan=() \ +DISAGREE facets=[] + +The marker field is `agree` when the two views match, `n/a(continuity)` for a conversation-scope +decision (deliberately excluded -- it answers a different question than the environment plan), and +the bare token `DISAGREE` when they do not. This sweep counts the DISAGREE token only. + +A DISAGREE line never fails a turn by design: the shadow must never break a run. That is exactly +why it needs a sweep. One hit is a real finding. + +TRIAGED EXCEPTIONS. Three line shapes are known SHADOW-COMPARATOR modeling gaps, not evictions: +the coordinator's behavior is correct and pinned by the runner's own tests, and only the shadow's +model of it disagrees (triage 2026-08-31, `f7-disagree-triage.md`; the comparator fixes are a +post-release follow-up). Without these exceptions the sweep fails on the runner's own expected +behavior on every window with real config traffic, and a check that cries wolf on healthy runs +stops being read. + +The exceptions are never silent. Every excluded line is printed with the shape that explains it +and the triage marker, the excluded count is reported separately from the verdict, and a DISAGREE +line matching NO triaged shape still fails. Each shape is anchored on both halves of the line +(decision and plan), so it cannot swallow a real disagreement that merely shares a reason. Delete +a shape when its comparator fix lands; `--no-exceptions` fails on every DISAGREE line and is how +you prove a shape can go. + +EXIT CODES + 0 PASS -- no unexplained DISAGREE line in the window. + 1 FAIL -- at least one line matched no triaged shape; the offending lines are printed. + 2 SKIP -- the runner log is not reachable (a remote deployment, or no docker). A SKIP prints + its reason and counts as a failure to explain, never as a green result. + + uv run sweep_disagree.py --since 2026-08-31T09:00:00 + uv run sweep_disagree.py --since 30m --container agenta-ee-dev-preview-runner-1 +""" + +import argparse +import os +import re +import subprocess +import sys +from urllib.parse import urlparse + +EXIT_PASS = 0 +EXIT_FAIL = 1 +EXIT_SKIP = 2 + +#: The marker written by `logReconcileShadow`. Both halves are required: `shadow ` keeps the sweep +#: off unrelated `[reconcile]` lines, and the padded ` DISAGREE ` token cannot match the word +#: DISAGREEMENTS that appears in the router's own source comments. +DISAGREE_LINE = re.compile(r"\[reconcile\] shadow .*\sDISAGREE\s") + +#: Printed beside every excluded line. An exception a reader cannot trace is indistinguishable +#: from a bug being hidden, so the provenance travels with the line, every time. +TRIAGE_MARKER = "known comparator gap, triaged 2026-08-31, see f7-disagree-triage.md" + +#: Shapes the 2026-08-31 triage proved are SHADOW-COMPARATOR modeling gaps, not evictions. +#: +#: In each one the coordinator's behavior is correct and pinned by the runner's own tests; only +#: the shadow's model of it disagrees. The comparator fixes are a post-release follow-up, so +#: without these exceptions the sweep fails on the runner's own expected behavior on every window +#: carrying real config traffic — and a check that cries wolf on healthy runs stops being read. +#: +#: Each entry is deliberately anchored on BOTH halves of the line, decision and plan. A shape that +#: matched on the decision alone would swallow real disagreements that happen to share a reason. +#: Delete an entry the moment its comparator fix lands; `--no-exceptions` is how you prove it has. +KNOWN_COMPARATOR_GAPS: tuple[tuple[str, str, "re.Pattern[str]"], ...] = ( + ( + "reopen-session-vs-config-rebuild", + "the plan names a reopen, but a reopen reinstalls the OLD config, so the coordinator's " + "rebuild is the only sound route (the 7x cluster)", + re.compile( + r"decision=rebuild\(mismatch:config\).*plan=reuse\(reopen-session\)" + ), + ), + ( + "approval-mismatch-under-environment-scope", + "the request carried no answer for the parked gate. That is a protocol fact, not an " + "environment fact, and the shadow call site labels it `environment`", + re.compile( + r"decision=rebuild\(approval-mismatch:unknown\).*plan=reuse\(no-op\)" + ), + ), + ( + "approval-resume-deferral", + "the approval branch never compares the fingerprint, by design; the re-park keeps the " + "old applied fingerprint, so the next idle turn rebuilds", + re.compile(r"decision=reuse\(approval-resume\).*plan=rebuild\("), + ), +) + + +def classify_disagree(line: str) -> tuple[str, str] | None: + """The triaged shape this DISAGREE line matches, as `(name, why)`, or None when it is new.""" + for name, why, pattern in KNOWN_COMPARATOR_GAPS: + if pattern.search(line): + return name, why + return None + + +def partition_known_gaps( + lines: list[str], use_exceptions: bool = True +) -> tuple[list[tuple[str, str, str]], list[str]]: + """Split DISAGREE lines into `(excluded, unexplained)`. + + `excluded` carries `(shape name, why, line)` so the caller can print each one with its + provenance. `unexplained` is what fails the sweep: a DISAGREE line matching no triaged shape + is exactly the drift this check exists to catch. + """ + excluded: list[tuple[str, str, str]] = [] + unexplained: list[str] = [] + for line in lines: + shape = classify_disagree(line) if use_exceptions else None + if shape is None: + unexplained.append(line) + else: + excluded.append((shape[0], shape[1], line)) + return excluded, unexplained + + +#: Hostnames that mean "this machine". Matched against the PARSED host, never the raw URL: a +#: substring test calls `https://runner-localhost.example` local, and the sweep would then scan +#: whatever runner happens to be on this box and report PASS for a deployment it never looked at. +LOCAL_HOSTS = frozenset({"localhost", "127.0.0.1", "0.0.0.0", "::1"}) + + +def base_host() -> str: + """The hostname `AGENTA_BASE` names, lowercased, or "" when it names none. + + Never raises. `urlparse(...).hostname` throws `ValueError` on a malformed bracketed IPv6 + literal such as `http://[::1`, and this helper decides whether the sweep runs at all — a + crash here would take the whole check down before it could even report a SKIP. + + The scheme-less fallback is IPv6-aware. Stripping the last colon-segment turns a bare `::1` + into `:` and `[::1]` into `[:`, so a real loopback would read as remote and the sweep would + skip a deployment it could have scanned. + """ + base = os.environ.get("AGENTA_BASE", "").strip() + try: + parsed = urlparse(base).hostname + except ValueError: + parsed = None + if parsed: + return parsed.strip().lower() + if "://" in base: + # It HAS a scheme and still yielded no hostname, so it is malformed (`http://[::1`). + # Falling through to the authority reader would answer "http", which is worse than + # admitting the base names no host. + return "" + + # No scheme, so `urlparse` put everything in `path`. Read the authority by hand: a bare + # `localhost:8480` is a normal way to set this. + authority = base.split("/")[0].split("@")[-1].strip() + if authority.startswith("["): + # A bracketed IPv6 literal, with or without a trailing `:port`. + return authority.split("]")[0].lstrip("[").strip().lower() + if authority.count(":") > 1: + # An UNbracketed IPv6 literal (`::1`). Every colon belongs to the address, so there is no + # port to strip. + return authority.lower() + return authority.rsplit(":", 1)[0].strip().lower() + + +def base_is_local() -> bool: + return base_host() in LOCAL_HOSTS + + +def autodetect_runner() -> tuple[str | None, str]: + """The runner container of the local stack, or `(None, why)` when docker cannot answer.""" + try: + out = subprocess.run( + ["docker", "ps", "--format", "{{.Names}}"], + capture_output=True, + text=True, + timeout=20, + ) + except (OSError, subprocess.SubprocessError) as e: + return None, f"`docker ps` failed: {e}" + if out.returncode != 0: + return None, f"`docker ps` exited {out.returncode}: {out.stderr.strip()[:200]}" + names = [n.strip() for n in out.stdout.splitlines() if n.strip()] + hits = [n for n in names if "runner" in n.lower()] + if not hits: + return None, "no container with `runner` in its name is running" + if len(hits) > 1: + return ( + None, + f"several runner containers are running ({', '.join(hits)}); pass --container", + ) + return hits[0], "" + + +def log_lines(container: str, since: str) -> tuple[list[str] | None, str]: + try: + out = subprocess.run( + ["docker", "logs", container, "--since", since], + capture_output=True, + text=True, + timeout=60, + ) + except (OSError, subprocess.SubprocessError) as e: + return None, f"`docker logs {container}` failed: {e}" + if out.returncode != 0: + return ( + None, + f"`docker logs {container}` exited {out.returncode}: {out.stderr.strip()[:200]}", + ) + return (out.stdout + out.stderr).splitlines(), "" + + +def main() -> int: + p = argparse.ArgumentParser( + description="Fail when the runner logged a reconciliation DISAGREE since a timestamp." + ) + p.add_argument( + "--since", + required=True, + help="start of the window: an ISO timestamp (2026-08-31T09:00:00) or a docker " + "duration (30m). Use the timestamp the gate session started.", + ) + p.add_argument( + "--container", + default=None, + help="runner container name. Default: autodetect the local stack's runner via `docker ps`.", + ) + p.add_argument( + "--no-exceptions", + action="store_true", + help="fail on EVERY DISAGREE line, including the triaged comparator gaps. Run this once " + "the comparator fixes land: a clean result is the proof that an exception can be deleted.", + ) + args = p.parse_args() + + if not base_is_local(): + base = os.environ.get("AGENTA_BASE", "") + print( + f"SKIP: the runner log is not reachable. AGENTA_BASE={base} is not a local " + "deployment, so this host has no docker socket for its runner. Read the log through " + "the operator channel for that deployment and grep for `[reconcile] shadow` lines " + "carrying the DISAGREE token.", + file=sys.stderr, + ) + return EXIT_SKIP + + container = args.container + if container is None: + container, why = autodetect_runner() + if container is None: + print(f"SKIP: cannot find the runner container: {why}", file=sys.stderr) + return EXIT_SKIP + + lines, why = log_lines(container, args.since) + if lines is None: + print(f"SKIP: cannot read the runner log: {why}", file=sys.stderr) + return EXIT_SKIP + + hits = [ln for ln in lines if DISAGREE_LINE.search(ln)] + excluded, unexplained = partition_known_gaps(hits, not args.no_exceptions) + + # Print every exclusion, always, with the shape that explains it. The count alone would let a + # growing cluster hide behind a number nobody reads. + if excluded: + print( + f"{len(excluded)} DISAGREE line(s) excluded as {TRIAGE_MARKER}:", + ) + for name, why, line in excluded: + print(f" [{name}] {line.strip()}") + print(f" why: {why}") + + if unexplained: + print( + f"FAIL: {len(unexplained)} unexplained reconciliation DISAGREE line(s) in " + f"{container} since {args.since}. The coordinator and the router disagree on session " + "identity in a shape no triage covers, which is the over-eviction signature." + ) + for line in unexplained: + print(f" {line.strip()}") + print( + f"({len(excluded)} further line(s) excluded as known comparator gaps.)" + if excluded + else "(no line matched a known comparator gap.)" + ) + return EXIT_FAIL + + print( + f"PASS: no unexplained reconciliation DISAGREE line in {container} since {args.since} " + f"({len(lines)} log lines scanned, {len(excluded)} known comparator gap(s) excluded)." + ) + return EXIT_PASS + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.agents/skills/agent-release-gate/resources/test_c5_key_blame_assertion.py b/.agents/skills/agent-release-gate/resources/test_c5_key_blame_assertion.py new file mode 100644 index 0000000000..d0b2b97e77 --- /dev/null +++ b/.agents/skills/agent-release-gate/resources/test_c5_key_blame_assertion.py @@ -0,0 +1,159 @@ +"""Unit test for C5's body-independent add-a-key assertion (run: `pytest +test_c5_key_blame_assertion.py`). + +The trap this pins: the first version of `matrix_c5_first_call_race.py` only failed when the +refusal BODY echoed the placeholder, so it could not see the path that matters most. Only the +litellm credits proxy echoes ("Received=dtn_****"). `api.anthropic.com` answers an unsubstituted +placeholder with "Invalid bearer token" and echoes nothing; OpenAI's echo is masked past the +literal `dtn_secret_`. On a direct provider the echo test is blind, and F6 shipped a user-blaming +401 straight through the cell that was supposed to catch exactly that. + +The assertion therefore keys on what is true regardless of body: this cell's sandbox is +necessarily fresh, so a credential refusal on it must never advise adding a key. The boundary has +to be narrow in both directions, so this tests both — a refusal that is NOT credential-shaped, and +a success, must not be dragged in. + +The error bodies are the ones captured live from the real providers during the F6 investigation +(2026-08-31), with a synthetic probe token; no real credential appears here. +""" + +import importlib +import sys +from pathlib import Path + +import pytest + + +def _c5(monkeypatch): + monkeypatch.setenv("AGENTA_BASE", "http://localhost:9999") + monkeypatch.setenv("AGENTA_PROJECT_ID", "proj-1") + monkeypatch.setenv("AGENTA_API_KEY", "test-key") + sys.path.insert(0, str(Path(__file__).resolve().parent)) + sys.modules.pop("qa_matrix_lib", None) + sys.modules.pop("matrix_c5_first_call_race", None) + return importlib.import_module("matrix_c5_first_call_race") + + +ANTHROPIC_BLAME = ( + "claude: model authentication failed — add the project's Anthropic key to the " + "project vault, or log in (OAuth). HTTP 401: Invalid bearer token" +) +OPENAI_BLAME = ( + "codex: model authentication failed — add the project's OpenAI key to the project " + "vault. HTTP 401: Incorrect API key provided: dtn_secr***************cdef" +) +PROXY_BLAME = ( + "HTTP 401: LiteLLM Virtual Key expected. Received=dtn_****. " + "Add your own provider key to keep going." +) +HONEST = ( + "A temporary issue kept this run's credentials from reaching the model. " + "Send the message again." +) + + +def test_anthropic_direct_blame_fails_without_any_echo(monkeypatch): + """The F6 case: no placeholder anywhere in the body, and it must still fail.""" + m = _c5(monkeypatch) + v = m.key_blame_verdict([], ANTHROPIC_BLAME, placeholder_seen=False) + assert v is not None + assert v["status"] == "FAIL" + assert "FRESH Daytona sandbox" in v["why"] + + +def test_openai_masked_echo_blame_fails(monkeypatch): + m = _c5(monkeypatch) + v = m.key_blame_verdict([], OPENAI_BLAME, placeholder_seen=False) + assert v["status"] == "FAIL" + + +def test_the_original_echo_case_still_fails_with_its_own_message(monkeypatch): + """Condition 1 keeps its stronger wording: the refusal itself proved the placeholder went out.""" + m = _c5(monkeypatch) + v = m.key_blame_verdict([], PROXY_BLAME, placeholder_seen=True) + assert v["status"] == "FAIL" + assert "placeholder refusal" in v["why"] + + +def test_a_starter_credits_code_counts_as_blame(monkeypatch): + m = _c5(monkeypatch) + v = m.key_blame_verdict( + ["starter_credits_exhausted"], "HTTP 401 Unauthorized", placeholder_seen=False + ) + assert v["status"] == "FAIL" + + +def test_the_honest_classification_is_not_blame(monkeypatch): + """With #6408 in, this is the expected outcome and must produce no verdict at all.""" + m = _c5(monkeypatch) + assert ( + m.key_blame_verdict( + ["credential_delivery_failed"], HONEST, placeholder_seen=False + ) + is None + ) + + +def test_a_non_credential_failure_is_never_dragged_in(monkeypatch): + """A timeout that happens to mention a key must not trip the assertion.""" + m = _c5(monkeypatch) + assert ( + m.key_blame_verdict( + [], "sandbox create timed out after 120s", placeholder_seen=False + ) + is None + ) + # Blame wording without a credential refusal: not this assertion's business. + assert ( + m.key_blame_verdict( + [], "please add the project's Anthropic key at your convenience", False + ) + is None + ) + + +def test_a_clean_run_produces_no_verdict(monkeypatch): + m = _c5(monkeypatch) + assert m.key_blame_verdict([], "", placeholder_seen=False) is None + + +@pytest.mark.parametrize( + "text", + [ + "HTTP 401: Invalid bearer token", + "HTTP 401 Unauthorized", + "authentication failed", + "authentication_error", + "invalid api key", + "invalid x-api-key", + ], +) +def test_auth_class_recognizes_every_provider_wording(monkeypatch, text): + m = _c5(monkeypatch) + assert m.AUTH_CLASS.search(text), text + + +def test_auth_class_does_not_match_a_bare_number(monkeypatch): + # `401` inside a longer number (a timestamp-derived id) is not a status code. The runner's + # own classifier guards the same way, and a false match here would fail honest runs. + m = _c5(monkeypatch) + assert not m.AUTH_CLASS.search("run id 1774014010 finished") + + +class TestPre6408: + """Against an older runner the assertion fails by construction; say so, never silently pass.""" + + def test_downgrades_the_body_independent_case_to_skip(self, monkeypatch): + m = _c5(monkeypatch) + v = m.key_blame_verdict( + [], ANTHROPIC_BLAME, placeholder_seen=False, pre_6408=True + ) + assert v["status"] == "SKIP" + assert "#6408" in v["why"] + + def test_never_softens_the_original_echo_case(self, monkeypatch): + # Every shipped version has been expected to catch an echoed placeholder. The flag + # excuses the new assertion only, never the one that already worked. + m = _c5(monkeypatch) + v = m.key_blame_verdict([], PROXY_BLAME, placeholder_seen=True, pre_6408=True) + assert v["status"] == "FAIL" diff --git a/.agents/skills/agent-release-gate/resources/test_check_secrets_teardown_pagination.py b/.agents/skills/agent-release-gate/resources/test_check_secrets_teardown_pagination.py new file mode 100644 index 0000000000..9e1cc10f00 --- /dev/null +++ b/.agents/skills/agent-release-gate/resources/test_check_secrets_teardown_pagination.py @@ -0,0 +1,135 @@ +"""Unit test for the Daytona Secrets cursor enumeration (run: `pytest +test_check_secrets_teardown_pagination.py`, or `uv run --no-sync pytest` from `api/`). + +The traps these pin, all found live against a ~3510-secret organization: + +1. The listing is CURSOR-paginated at 100 per response. An unpaginated read sees only the first + 100 of ~3510, so the before/after set difference the cell is built on becomes noise in both + directions: it invents leftovers and hides real ones at the same time. +2. A `page` parameter is SILENTLY IGNORED — the same 100 ids come back for every "page". Code + written against `page` therefore loops forever on identical data while looking like it is + making progress. The enumeration must notice a cursor that stops advancing. +3. A huge organization must not be able to hang the cell, so both a page ceiling and a + wall-clock budget bound the walk. Hitting either is a SKIP, never a guess. + +These use the `fetch` seam, so no request reaches Daytona and NO SECRET IS EVER CREATED, READ AT +VALUE, OR DELETED. Names and ids here are synthetic. +""" + +import importlib +import sys +from pathlib import Path + +import pytest + + +def _mod(monkeypatch): + monkeypatch.setenv("AGENTA_BASE", "https://qa.example") + monkeypatch.setenv("AGENTA_PROJECT_ID", "proj-1") + monkeypatch.setenv("AGENTA_API_KEY", "test-key") + sys.path.insert(0, str(Path(__file__).resolve().parent)) + sys.modules.pop("qa_matrix_lib", None) + sys.modules.pop("check_secrets_teardown", None) + return importlib.import_module("check_secrets_teardown") + + +def _page(start: int, count: int, next_cursor): + """One synthetic response page, in the shape `ListSecretsResponse` declares.""" + return { + "items": [ + {"id": f"id-{i}", "name": f"agenta_{i:036x}_0"} + for i in range(start, start + count) + ], + "total": 237, + "nextCursor": next_cursor, + } + + +def test_follows_the_cursor_to_exhaustion(monkeypatch): + """Three pages (100, 100, 37) must yield all 237, not just the first 100.""" + m = _mod(monkeypatch) + pages = { + None: _page(0, 100, "cur-1"), + "cur-1": _page(100, 100, "cur-2"), + "cur-2": _page(200, 37, None), + } + asked = [] + + def fetch(cursor): + asked.append(cursor) + return pages[cursor] + + out = m.list_secret_ids_by_name(fetch=fetch) + assert len(out) == 237 + assert asked == [None, "cur-1", "cur-2"] + assert out[f"agenta_{236:036x}_0"] == "id-236" + + +def test_stops_on_a_null_next_cursor(monkeypatch): + m = _mod(monkeypatch) + out = m.list_secret_ids_by_name(fetch=lambda cursor: _page(0, 4, None)) + assert len(out) == 4 + + +def test_missing_next_cursor_field_terminates(monkeypatch): + """An absent `nextCursor` must end the walk, not be read as 'keep going'.""" + m = _mod(monkeypatch) + out = m.list_secret_ids_by_name( + fetch=lambda cursor: {"items": [{"id": "id-0", "name": "agenta_x_0"}]} + ) + assert out == {"agenta_x_0": "id-0"} + + +def test_a_repeating_cursor_is_refused_not_looped(monkeypatch): + """The ignored-`page` shape: the same page forever. Must SKIP, not hang.""" + m = _mod(monkeypatch) + calls = [] + + def fetch(cursor): + calls.append(cursor) + return _page(0, 100, "same-cursor") + + with pytest.raises(m.SkipCheck) as e: + m.list_secret_ids_by_name(fetch=fetch) + assert "stopped advancing" in str(e.value) + # It must give up almost immediately, not walk to the page ceiling. + assert len(calls) == 2 + + +def test_page_ceiling_is_a_skip_not_a_partial_answer(monkeypatch): + """A cursor that advances forever must stop at the ceiling and refuse to answer.""" + m = _mod(monkeypatch) + monkeypatch.setattr(m, "MAX_PAGES", 5) + n = iter(range(10_000)) + + def fetch(cursor): + i = next(n) + return _page(i * 100, 100, f"cur-{i}") + + with pytest.raises(m.SkipCheck) as e: + m.list_secret_ids_by_name(fetch=fetch) + assert "page ceiling" in str(e.value) + + +def test_time_budget_is_a_skip(monkeypatch): + m = _mod(monkeypatch) + monkeypatch.setattr(m, "MAX_ENUMERATION_SECONDS", -1.0) + with pytest.raises(m.SkipCheck) as e: + m.list_secret_ids_by_name(fetch=lambda cursor: _page(0, 1, "cur-1")) + assert "exceeded" in str(e.value) + + +def test_a_malformed_payload_is_a_skip(monkeypatch): + m = _mod(monkeypatch) + with pytest.raises(m.SkipCheck): + m.list_secret_ids_by_name(fetch=lambda cursor: {"items": "not-a-list"}) + + +def test_only_this_runs_generated_names_are_treated_as_created(monkeypatch): + """The name filter must accept the runner's generated shape and reject anything else.""" + m = _mod(monkeypatch) + assert m.RUN_SECRET_NAME.match("agenta_" + "a" * 36 + "_0") + assert m.RUN_SECRET_NAME.match("agenta_" + "0" * 36 + "_12") + assert not m.RUN_SECRET_NAME.match("agenta_short_0") + assert not m.RUN_SECRET_NAME.match("openai-key") + assert not m.RUN_SECRET_NAME.match("agenta_" + "a" * 36) diff --git a/.agents/skills/agent-release-gate/resources/test_gate_review_followups.py b/.agents/skills/agent-release-gate/resources/test_gate_review_followups.py new file mode 100644 index 0000000000..0a61e24c74 --- /dev/null +++ b/.agents/skills/agent-release-gate/resources/test_gate_review_followups.py @@ -0,0 +1,648 @@ +"""Tests for the CodeRabbit review follow-ups on the gate checks (#6402). + +Each class pins one finding. The theme running through them: a check that turns a real failure +into a green SKIP, or infers a strong claim from missing evidence, is worse than no check — it +spends a reviewer's trust and returns nothing for it. +""" + +import importlib +import sys +from pathlib import Path + +import pytest + + +def _mod(monkeypatch, name): + monkeypatch.setenv("AGENTA_BASE", "http://localhost:9999") + monkeypatch.setenv("AGENTA_PROJECT_ID", "proj-1") + monkeypatch.setenv("AGENTA_API_KEY", "test-key") + monkeypatch.setenv("DAYTONA_API_KEY", "test-daytona-key") + sys.path.insert(0, str(Path(__file__).resolve().parent)) + for cached in ("qa_matrix_lib", name): + sys.modules.pop(cached, None) + return importlib.import_module(name) + + +class TestNarrowSkipClassification: + """A transport failure must never become a green SKIP (CodeRabbit, major).""" + + REAL_FAILURES = [ + "ECONNRESET while contacting the credential service", + "connection error: upstream refused", + "credential_delivery_failed: the run's credentials did not reach the model", + "the request timed out after 120s", + "502 Bad Gateway from the connection broker", + ] + VAULT_DIAGNOSTICS = [ + "connection 'x' not found for provider 'anthropic'", + "multiple connections for provider 'openai'", + "no connections for provider 'anthropic'", + ] + + @pytest.mark.parametrize("text", REAL_FAILURES) + def test_teardown_keeps_a_real_failure(self, monkeypatch, text): + m = _mod(monkeypatch, "check_secrets_teardown") + assert m.environment_cause(text) is None, text + + @pytest.mark.parametrize("text", VAULT_DIAGNOSTICS) + def test_teardown_still_skips_a_real_vault_miss(self, monkeypatch, text): + m = _mod(monkeypatch, "check_secrets_teardown") + assert m.environment_cause(text) is not None, text + + def test_a_transport_failure_wins_over_a_vault_phrase_beside_it(self, monkeypatch): + # The mixed case: a credential word next to a real error must not excuse the error. + m = _mod(monkeypatch, "check_secrets_teardown") + assert ( + m.environment_cause( + "ECONNRESET; also: no connections for provider 'anthropic'" + ) + is None + ) + + @pytest.mark.parametrize("text", REAL_FAILURES) + def test_c5_keeps_a_real_failure(self, monkeypatch, text): + m = _mod(monkeypatch, "matrix_c5_first_call_race") + assert m.missing_vault_credential(text) is False, text + + @pytest.mark.parametrize("text", VAULT_DIAGNOSTICS) + def test_c5_still_skips_a_real_vault_miss(self, monkeypatch, text): + m = _mod(monkeypatch, "matrix_c5_first_call_race") + assert m.missing_vault_credential(text) is True, text + + +class TestMixedCauses: + """SKIP only when EVERY error frame is an environment cause (CodeRabbit, major).""" + + def test_a_credit_frame_does_not_excuse_a_transport_frame(self, monkeypatch): + m = _mod(monkeypatch, "check_secrets_teardown") + frames = [ + "Your free Agenta credits are used up.", + "ECONNRESET talking to the sandbox", + ] + unexplained = [e for e in frames if m.environment_cause(e) is None] + assert unexplained == ["ECONNRESET talking to the sandbox"] + + def test_all_credit_frames_are_still_a_skip(self, monkeypatch): + m = _mod(monkeypatch, "check_secrets_teardown") + frames = [ + "Your free Agenta credits are used up.", + "the model provider account has insufficient credit", + ] + assert [e for e in frames if m.environment_cause(e) is None] == [] + + +class TestHttpsGuard: + """A bearer token must never go over cleartext (CodeRabbit, security).""" + + def test_an_http_base_is_refused_before_any_request(self, monkeypatch): + m = _mod(monkeypatch, "check_secrets_teardown") + monkeypatch.setenv("DAYTONA_API_URL", "http://daytona.internal/api") + with pytest.raises(m.SkipCheck) as e: + m.daytona_api_url() + assert "not HTTPS" in str(e.value) + + def test_an_https_base_is_accepted(self, monkeypatch): + m = _mod(monkeypatch, "check_secrets_teardown") + monkeypatch.setenv("DAYTONA_API_URL", "https://app.daytona.io/api") + assert m.daytona_api_url() == "https://app.daytona.io/api" + + +class TestMalformedPayloadShapes: + """A malformed page is a SKIP naming the shape, never a stack trace (CodeRabbit, minor).""" + + def test_a_non_object_body_is_a_skip(self, monkeypatch): + m = _mod(monkeypatch, "check_secrets_teardown") + for body in (None, [], "nope", 7): + with pytest.raises(m.SkipCheck): + m.list_secret_ids_by_name(fetch=lambda cursor, b=body: b) + + def test_an_unhashable_next_cursor_is_a_skip(self, monkeypatch): + # `cursor in seen_cursors` would raise TypeError on a list, aborting the whole gate. + m = _mod(monkeypatch, "check_secrets_teardown") + for bad in ([], {}, 7): + with pytest.raises(m.SkipCheck): + m.list_secret_ids_by_name( + fetch=lambda cursor, c=bad: {"items": [], "nextCursor": c} + ) + + +class TestSweepHostParsing: + """`localhost` as a substring is not a local deployment (CodeRabbit, major).""" + + @pytest.mark.parametrize( + "base,expected", + [ + ("http://localhost:8480", True), + ("http://127.0.0.1:9", True), + ("localhost:8480", True), + ("https://runner-localhost.example", False), + ("https://localhost.attacker.com", False), + ("https://cloud.agenta.ai", False), + ("", False), + ], + ) + def test_only_a_real_local_host_counts(self, monkeypatch, base, expected): + monkeypatch.setenv("AGENTA_BASE", base) + sys.path.insert(0, str(Path(__file__).resolve().parent)) + sys.modules.pop("sweep_disagree", None) + m = importlib.import_module("sweep_disagree") + assert m.base_is_local() is expected, base + + +class TestLedgerAvailability: + """An unanswered ledger query is not proof that nothing was stored (CodeRabbit, major).""" + + def test_the_library_reports_availability_separately(self, monkeypatch): + lib = _mod(monkeypatch, "qa_matrix_lib") + + class _Resp: + status_code = 503 + + def json(self): # pragma: no cover - never reached on a 503 + return {} + + monkeypatch.setattr(lib, "api_call", lambda *a, **k: _Resp()) + rows, available = lib.turn_ledger_or_unavailable("s") + assert rows == [] + assert available is False + # The old signature cannot tell the caller which of the two it got. + assert lib.turn_ledger("s") == [] + + def test_an_empty_but_answered_ledger_is_available(self, monkeypatch): + lib = _mod(monkeypatch, "qa_matrix_lib") + + class _Resp: + status_code = 200 + + def json(self): + return {"turns": []} + + monkeypatch.setattr(lib, "api_call", lambda *a, **k: _Resp()) + assert lib.turn_ledger_or_unavailable("s") == ([], True) + + +class TestH1Probe: + """`probe`'s two new decisions, driven through the five outcomes verified by hand. + + Lifted from the cross-reviewer's driver. The decisions under test are (a) execution evidence + outranks a later refusal, and (b) an unanswered ledger cannot support a PASS that asserts + nothing was stored. Both were correct but untested, which is how the first version of this + rule shipped with a hole in it. + """ + + ERROR_FRAME = [ + { + "type": "data-agent-error", + "data": { + "code": "agent_run_failed", + "errorText": "harness kind is invalid", + }, + } + ] + + class FakeTurn: + def __init__(self, reply="", raw=None, errors=None): + self.reply = reply + self.frames = [f.get("type", "") for f in (raw or [])] + self.raw_frames = raw or [] + self.tool_calls = [] + self.errors = errors or [] + + def _drive( + self, + monkeypatch, + *, + commit_status, + turn, + rows, + available, + harness=None, + ): + import types + + h1 = _mod(monkeypatch, "matrix_h1_bad_harness") + monkeypatch.setattr( + h1, + "commit_direct", + lambda *a, **k: types.SimpleNamespace( + status_code=commit_status, + text='{"detail":{"code":"invalid_harness_kind","message":"invalid harness.kind"}}', + ), + ) + monkeypatch.setattr(h1, "invoke", lambda *a, **k: turn) + monkeypatch.setattr( + h1, "turn_ledger_or_unavailable", lambda *a, **k: (rows, available) + ) + monkeypatch.setattr(h1.time, "sleep", lambda *a: None) + return h1.probe("wf", "var", {}, "case", harness or {"kind": 12345}) + + def test_1_commit_refusal_with_nothing_stored_passes(self, monkeypatch): + d = self._drive( + monkeypatch, + commit_status=422, + turn=self.FakeTurn(), + rows=[], + available=True, + ) + assert d["status"] == "PASS", d["why"] + assert d["refused_by"] == "commit_api" + + def test_2_runner_stream_error_with_nothing_stored_passes(self, monkeypatch): + d = self._drive( + monkeypatch, + commit_status=200, + turn=self.FakeTurn(raw=self.ERROR_FRAME, errors=["harness kind bad"]), + rows=[], + available=True, + ) + assert d["status"] == "PASS", d["why"] + assert d["refused_by"] == "runner_stream" + + def test_3_output_plus_a_streamed_error_fails(self, monkeypatch): + # THE BUG THIS RULE FIXES. The old condition required `not error_text`, so a streamed + # refusal arriving after the turn had already spoken let a defaulted run pass. + d = self._drive( + monkeypatch, + commit_status=200, + turn=self.FakeTurn( + reply="READY", raw=self.ERROR_FRAME, errors=["harness kind bad"] + ), + rows=[], + available=True, + ) + assert d["status"] == "FAIL", d["why"] + assert "RAN" in d["why"] + + def test_4_stored_harness_kind_with_no_output_fails(self, monkeypatch): + # The same evidence read from the other side: the row proves a turn was persisted under a + # real harness even though the stream said nothing. + d = self._drive( + monkeypatch, + commit_status=200, + turn=self.FakeTurn(raw=self.ERROR_FRAME, errors=["e"]), + rows=[{"harness_kind": "pi_core"}], + available=True, + ) + assert d["status"] == "FAIL", d["why"] + assert "pi_core" in d["why"] + + def test_5_unanswered_ledger_with_a_refusal_present_fails(self, monkeypatch): + # A refusal alone is not enough: this PASS asserts nothing was stored, and a query that + # never answered cannot support that. + d = self._drive( + monkeypatch, + commit_status=422, + turn=self.FakeTurn(), + rows=[], + available=False, + ) + assert d["status"] == "FAIL", d["why"] + assert "did not answer" in d["why"] + + +class TestSF2IsNamedNotSoftened: + """The cleared-harness FAIL is real, and it is the filed finding SF2 (W5 handling).""" + + def test_the_cleared_harness_fail_names_sf2(self, monkeypatch): + probe = TestH1Probe() + d = probe._drive( + monkeypatch, + commit_status=200, + turn=probe.FakeTurn(reply="READY"), + rows=[{"harness_kind": "pi_core"}], + available=True, + harness={"kind": None}, + ) + # Still a FAIL. The invariant really is broken; only the reader's context improves. + assert d["status"] == "FAIL" + assert d["known_finding"] == "SF2" + assert "silently defaults to pi_core" in d["why"] + assert "filed for the next release" in d["why"] + + def test_a_wrong_type_harness_that_runs_is_not_sf2(self, monkeypatch): + # A different, UNFILED defect must read as new breakage, not borrow SF2's excuse. + probe = TestH1Probe() + d = probe._drive( + monkeypatch, + commit_status=200, + turn=probe.FakeTurn(reply="READY"), + rows=[{"harness_kind": "pi_core"}], + available=True, + harness={"kind": 12345}, + ) + assert d["status"] == "FAIL" + assert "known_finding" not in d + assert "SF2" not in d["why"] + + def test_a_cleared_harness_defaulting_elsewhere_is_not_sf2(self, monkeypatch): + # SF2 is specifically the pi_core default. A cleared harness running as claude would be a + # different finding and must not be filed under this one. + probe = TestH1Probe() + d = probe._drive( + monkeypatch, + commit_status=200, + turn=probe.FakeTurn(reply="READY"), + rows=[{"harness_kind": "claude"}], + available=True, + harness={"kind": None}, + ) + assert d["status"] == "FAIL" + assert "known_finding" not in d + + +class TestSettleDeadlineBoundsEachProbe: + """The deadline must bound the probes themselves, not only the gaps (CodeRabbit, major).""" + + def test_each_probe_gets_the_remaining_budget_as_its_timeout(self, monkeypatch): + m = _mod(monkeypatch, "check_secrets_teardown") + seen: list[float] = [] + + def fake_get(path, params=None, timeout=30.0): + seen.append(timeout) + + class _R: + status_code = 404 + + return _R() + + monkeypatch.setattr(m, "_get", fake_get) + assert m.secret_exists("id-1", timeout=3.0) is False + assert seen == [3.0] + + def test_a_probe_cannot_be_given_more_than_the_settle_budget(self, monkeypatch): + # A fixed 30s probe inside a 1s budget could return long after the deadline and let the + # loop claim a within-budget deletion it never observed within budget. + m = _mod(monkeypatch, "check_secrets_teardown") + assert m.INITIAL_PROBE_SECONDS <= 10.0 + assert m.SETTLE_POLL_SECONDS <= 5.0 + + +class TestC5TreatsDeliveryFailureAsReal: + """A delivery failure is what C5 tests, so it can never excuse a SKIP (CodeRabbit, major).""" + + def test_a_textual_delivery_failure_is_not_a_vault_miss(self, monkeypatch): + m = _mod(monkeypatch, "matrix_c5_first_call_race") + for text in [ + "credential_delivery_failed: no connections for provider 'anthropic'", + "A temporary issue kept this run's credentials from reaching the model.", + ]: + assert m.missing_vault_credential(text) is False, text + + def test_the_two_cells_agree_on_what_counts_as_a_real_failure(self, monkeypatch): + # These lists drifted once already: the teardown check carried the delivery markers and + # C5 did not, which is exactly how the SKIP hole opened. + c5 = _mod(monkeypatch, "matrix_c5_first_call_race") + teardown = _mod(monkeypatch, "check_secrets_teardown") + assert set(c5.TRANSPORT_FAILURE_MARKERS) == set( + teardown.TRANSPORT_FAILURE_MARKERS + ) + + def test_a_plain_vault_miss_still_skips(self, monkeypatch): + m = _mod(monkeypatch, "matrix_c5_first_call_race") + assert m.missing_vault_credential("no connections for provider 'anthropic'") + + +class TestLedgerPayloadValidation: + """A 200 is not an answer until the payload has the promised shape (CodeRabbit, major).""" + + def _lib_returning(self, monkeypatch, payload, status=200): + lib = _mod(monkeypatch, "qa_matrix_lib") + + class _Resp: + status_code = status + + def json(self): + if isinstance(payload, Exception): + raise payload + return payload + + monkeypatch.setattr(lib, "api_call", lambda *a, **k: _Resp()) + return lib + + @pytest.mark.parametrize( + "payload", + [ + {}, + {"turns": None}, + {"turns": "nope"}, + {"turns": [1, 2]}, + [], + "nope", + None, + ValueError("not json"), + ], + ) + def test_a_malformed_200_is_unavailable(self, monkeypatch, payload): + # The load-bearing one: `{}` and `{"turns": null}` would otherwise be ([], True), which + # h1 turns into a PASS asserting nothing was stored. + lib = self._lib_returning(monkeypatch, payload) + assert lib.turn_ledger_or_unavailable("s") == ([], False), payload + + def test_a_well_formed_empty_ledger_is_available(self, monkeypatch): + lib = self._lib_returning(monkeypatch, {"turns": []}) + assert lib.turn_ledger_or_unavailable("s") == ([], True) + + def test_rows_are_returned_when_the_shape_is_right(self, monkeypatch): + lib = self._lib_returning(monkeypatch, {"turns": [{"sandbox_id": "sb-1"}]}) + rows, available = lib.turn_ledger_or_unavailable("s") + assert available is True + assert rows == [{"sandbox_id": "sb-1"}] + + def test_the_back_compat_wrapper_still_returns_a_list(self, monkeypatch): + lib = self._lib_returning(monkeypatch, {}) + assert lib.turn_ledger("s") == [] + + def test_h1_fails_on_a_malformed_200_rather_than_passing(self, monkeypatch): + # End to end: the seam change is only worth anything if h1 refuses the PASS. + probe = TestH1Probe() + d = probe._drive( + monkeypatch, + commit_status=422, + turn=probe.FakeTurn(), + rows=[], + available=False, + ) + assert d["status"] == "FAIL" + assert "did not answer" in d["why"] + + +class TestSweepIPv6Hosts: + """The base parser must not crash, and must not mangle IPv6 (CodeRabbit, two minors).""" + + def _host(self, monkeypatch, base): + monkeypatch.setenv("AGENTA_BASE", base) + sys.path.insert(0, str(Path(__file__).resolve().parent)) + sys.modules.pop("sweep_disagree", None) + return importlib.import_module("sweep_disagree") + + def test_a_malformed_bracketed_ipv6_does_not_crash(self, monkeypatch): + # `urlparse("http://[::1").hostname` raises ValueError; this helper decides whether the + # sweep runs at all, so a crash here takes the check down before it can even SKIP. + m = self._host(monkeypatch, "http://[::1") + assert m.base_is_local() is False + assert m.base_host() == "" + + @pytest.mark.parametrize( + "base,expected", + [ + ("http://[::1]:8480", True), + ("http://[::1]", True), + ("[::1]:8480", True), + ("[::1]", True), + ("::1", True), + ("http://localhost:8480", True), + ("localhost:8480", True), + ("https://runner-localhost.example", False), + ("http://[2001:db8::1]:8480", False), + ("2001:db8::1", False), + ("", False), + ], + ) + def test_ipv6_and_named_hosts_classify_correctly(self, monkeypatch, base, expected): + m = self._host(monkeypatch, base) + assert m.base_is_local() is expected, f"{base} -> {m.base_host()!r}" + + +class TestAbsenceObservedAfterTheDeadline: + """An absence first SEEN after the budget is not an in-budget deletion (CodeRabbit, major). + + The first probe carries a floor so it can complete at all, which means on a short budget it + can itself finish after the deadline. Without timestamping the observation, that floor quietly + reopened the deadline hole the polling fix had just closed: a slow 404 came back, the loop saw + an empty leftover, and the cell reported `deleted within 0s` — a number nobody measured. + """ + + RUN_SECRET = "agenta_" + "a" * 36 + "_0" + + def _drive(self, monkeypatch, *, settle, probe_seconds, present=False): + m = _mod(monkeypatch, "check_secrets_teardown") + + clock = {"now": 0.0} + monkeypatch.setattr(m.time, "monotonic", lambda: clock["now"]) + monkeypatch.setattr( + m.time, "sleep", lambda s: clock.__setitem__("now", clock["now"] + s) + ) + + def slow_secret_exists(secret_id, timeout=30.0): + # Every probe costs wall-clock time, which is the whole point: a probe is not free + # and can outlast the budget it was meant to respect. + clock["now"] += probe_seconds + return present + + monkeypatch.setattr(m, "secret_exists", slow_secret_exists) + + inventories = [{}, {self.RUN_SECRET: "id-1"}] + monkeypatch.setattr( + m, "list_secret_ids_by_name", lambda *a, **k: inventories.pop(0) + ) + monkeypatch.setattr(m, "create_workflow", lambda *a, **k: ("wf-1", "var-1")) + monkeypatch.setattr(m, "seed_and_baseline", lambda *a, **k: ("rev-1", 1)) + monkeypatch.setattr(m, "refs", lambda *a, **k: {}) + monkeypatch.setattr(m, "archive", lambda *a, **k: None) + + class _Turn: + errors: list = [] + + monkeypatch.setattr(m, "invoke", lambda *a, **k: _Turn()) + return m + + def test_settle_zero_with_a_slow_404_does_not_pass(self, monkeypatch): + # CodeRabbit's exact regression: the Secret IS gone, but the read that proved it landed + # 2s into a 0s budget. + m = self._drive(monkeypatch, settle=0, probe_seconds=2.0) + with pytest.raises(m.SkipCheck) as e: + m.secrets_teardown(0) + why = str(e.value) + assert "AFTER the 0s settle budget" in why + assert "unproven" in why + # And it says plainly that nothing leaked, so a reader does not chase a leak that is not + # there: this is a measurement gap, not a violated invariant. + assert "No Secret outlived its run" in why + + def test_a_short_probe_inside_a_real_budget_still_passes(self, monkeypatch): + # The positive control. Without it, "never PASS" would satisfy the test above. + m = self._drive(monkeypatch, settle=60, probe_seconds=2.0) + r = m.secrets_teardown(60) + assert r["status"] == "PASS", r["why"] + assert "deleted within 60s" in r["why"] + + def test_a_secret_that_really_survives_still_fails(self, monkeypatch): + # The distinction that matters: an unproven-but-clean run is a SKIP, a genuine leftover is + # a FAIL. Conflating them either way would break the check. + m = self._drive(monkeypatch, settle=1, probe_seconds=2.0, present=True) + r = m.secrets_teardown(1) + assert r["status"] == "FAIL", r + assert self.RUN_SECRET in r["leftover_secret_names"] + + +class TestRowPresenceIsTheEvidence: + """A stored row refutes PASS even when its harness_kind is unset (CodeRabbit, major). + + `stored_harnesses` keeps only truthy values, so a row with a missing, null or empty + `harness_kind` collapsed to `[]` and the probe reached PASS with a turn demonstrably + persisted. Same class as the ledger-availability finding: an absent FIELD was read as an + absent THING. + """ + + @pytest.mark.parametrize( + "row", + [ + {"harness_kind": None}, + {"harness_kind": ""}, + {}, + {"sandbox_id": "sb-1"}, + ], + ) + def test_a_row_without_a_usable_kind_still_fails(self, monkeypatch, row): + probe = TestH1Probe() + d = probe._drive( + monkeypatch, + commit_status=422, + turn=probe.FakeTurn(), + rows=[row], + available=True, + ) + assert d["status"] == "FAIL", d["why"] + assert "stored_turn_rows=1" in d["why"] + + def test_such_a_row_is_not_labelled_sf2(self, monkeypatch): + # SF2 is specifically the silent pi_core default. An unset stored kind proves a turn ran + # but not what it ran AS, so it must read as new breakage rather than the filed finding. + probe = TestH1Probe() + d = probe._drive( + monkeypatch, + commit_status=422, + turn=probe.FakeTurn(), + rows=[{"harness_kind": None}], + available=True, + harness={"kind": None}, + ) + assert d["status"] == "FAIL" + assert "known_finding" not in d + assert "SF2" not in d["why"] + + def test_output_with_no_stored_kind_is_not_labelled_sf2_either(self, monkeypatch): + # A cleared harness that visibly ran but stored no kind: still a FAIL, still unlabelled. + probe = TestH1Probe() + d = probe._drive( + monkeypatch, + commit_status=200, + turn=probe.FakeTurn(reply="READY"), + rows=[], + available=True, + harness={"kind": None}, + ) + assert d["status"] == "FAIL" + assert "known_finding" not in d + + def test_an_empty_answered_ledger_with_no_output_still_passes(self, monkeypatch): + # The positive control: row presence is the evidence, so NO rows must still allow a PASS. + probe = TestH1Probe() + d = probe._drive( + monkeypatch, + commit_status=422, + turn=probe.FakeTurn(), + rows=[], + available=True, + ) + assert d["status"] == "PASS", d["why"] diff --git a/.agents/skills/agent-release-gate/resources/test_out_of_credit_skip.py b/.agents/skills/agent-release-gate/resources/test_out_of_credit_skip.py new file mode 100644 index 0000000000..19d1b778b2 --- /dev/null +++ b/.agents/skills/agent-release-gate/resources/test_out_of_credit_skip.py @@ -0,0 +1,207 @@ +"""Unit test for the out-of-credit classification boundary (run: `pytest +test_out_of_credit_skip.py`, or `uv run --no-sync pytest` from `api/`). + +The trap this pins: an exhausted provider key is an ENVIRONMENT condition, and both incident +cells used to render it in the shape their docstrings reserve for a real defect -- C5 as "turn +failed for another reason", check_secrets_teardown as "the journey never ran". That costs a +reviewer's attention on a topped-up balance, and worse, it teaches the reader that these cells' +FAILs are sometimes noise, which is how a genuine regression gets waved through later. + +The boundary has to be narrow in BOTH directions, so this tests both: + + a credit or billing signature -> SKIP "environment: provider key out of credit" + anything else -> the FAIL is kept + +In particular a bare 401, a rate limit, and the placeholder refusal the whole C5 cell exists for +must all stay FAIL. Nothing here reaches a network; the cells are driven with stubs. +""" + +import importlib +import sys +from pathlib import Path + +import pytest + +CREDIT_ERRORS = [ + # The runner's own user-facing credits copy, both variants. + "Your free Agenta credits are used up. Add your own provider key to keep going.", + "Free Agenta credits are paused right now. Add your own provider key to continue.", + "Agenta credits are temporarily unavailable. Try again in a moment.", + # The provider's billing refusal, which the runner classifies as a plain `runner_error`, so + # the code alone cannot catch it. + "claude: the model provider account has insufficient credit (check ANTHROPIC_API_KEY).", + "RateLimitError: You exceeded your current quota, please check your plan and billing", + "insufficient_quota", + "your credit balance is too low to access the Anthropic API", + "no credits remaining on this key", + # LiteLLM's admission-time refusal. + "ExceededBudget: Crossed spend within budget_exceeded for key", +] + +NON_CREDIT_ERRORS = [ + # A bare auth failure. The key may be perfectly funded and simply wrong. + "claude: model authentication failed -- add ANTHROPIC_API_KEY to the project vault.", + "HTTP 401: Unauthorized", + # Throttling is not a billing stop, and calling it one is the exact confusion errors.ts + # warns about. + "Too many requests right now. Try again in a moment.", + "rate_limit_error: please slow down", + # The placeholder race. This is the defect C5 exists to catch; it must never become a SKIP. + "LiteLLM Virtual Key expected. Received=dtn_secret_abc123", # gitleaks:allow + "A temporary issue kept this run's credentials from reaching the model. Send the message again.", + # Ordinary failures. + "agent run failed", + "sandbox create timed out after 120s", + "", +] + + +def _lib(monkeypatch): + monkeypatch.setenv("AGENTA_BASE", "http://localhost:9999") + monkeypatch.setenv("AGENTA_PROJECT_ID", "proj-1") + monkeypatch.setenv("AGENTA_API_KEY", "test-key") + sys.path.insert(0, str(Path(__file__).resolve().parent)) + sys.modules.pop("qa_matrix_lib", None) + return importlib.import_module("qa_matrix_lib") + + +# --------------------------------------------------------------------------- the classifier + + +@pytest.mark.parametrize("text", CREDIT_ERRORS) +def test_credit_signatures_are_recognized(monkeypatch, text): + m = _lib(monkeypatch) + assert m.out_of_credit(text) == "environment: provider key out of credit" + + +@pytest.mark.parametrize("text", NON_CREDIT_ERRORS) +def test_everything_else_is_not_a_credit_failure(monkeypatch, text): + m = _lib(monkeypatch) + assert m.out_of_credit(text) is None + + +@pytest.mark.parametrize( + "code", ["starter_credits_exhausted", "starter_credits_program_paused"] +) +def test_starter_credit_codes_are_recognized_without_prose(monkeypatch, code): + m = _lib(monkeypatch) + reason = m.out_of_credit("", [code]) + assert reason == f"environment: provider key out of credit ({code})" + + +def test_an_unrelated_code_is_not_a_credit_failure(monkeypatch): + m = _lib(monkeypatch) + assert m.out_of_credit("agent run failed", ["runner_error"]) is None + assert m.out_of_credit("", ["credential_delivery_failed"]) is None + + +# --------------------------------------------------------------------------- the cells + + +class _Turn: + """The parts of `qa_matrix_lib.Turn` the two cells read.""" + + def __init__(self, errors=(), frames=(), reply=""): + self.errors = list(errors) + self.raw_frames = list(frames) + self.frames = [f.get("type", "") for f in self.raw_frames] + self.reply = reply + self.tool_calls = [] + + +def _c5(monkeypatch, turn): + monkeypatch.setenv("AGENTA_BASE", "http://localhost:9999") + monkeypatch.setenv("AGENTA_PROJECT_ID", "proj-1") + monkeypatch.setenv("AGENTA_API_KEY", "test-key") + sys.path.insert(0, str(Path(__file__).resolve().parent)) + sys.modules.pop("qa_matrix_lib", None) + sys.modules.pop("matrix_c5_first_call_race", None) + mod = importlib.import_module("matrix_c5_first_call_race") + monkeypatch.setattr(mod, "create_workflow", lambda *a, **k: ("wf-1", "var-1")) + monkeypatch.setattr(mod, "seed_and_baseline", lambda *a, **k: ("rev-1", 1)) + monkeypatch.setattr(mod, "refs", lambda *a, **k: {}) + monkeypatch.setattr(mod, "archive", lambda *a, **k: None) + monkeypatch.setattr(mod, "invoke", lambda *a, **k: turn) + monkeypatch.setattr(mod, "turn_ledger", lambda *a, **k: [{"sandbox_id": "sb-1"}]) + monkeypatch.setattr( + mod, "count_proxy_placeholder_refusals", lambda *a, **k: (None, "n/a") + ) + monkeypatch.setattr(mod.time, "sleep", lambda *a: None) + return mod + + +def test_c5_skips_an_exhausted_key(monkeypatch): + turn = _Turn( + errors=["claude: the model provider account has insufficient credit (check X)."] + ) + mod = _c5(monkeypatch, turn) + r = mod.c5_first_call_race() + assert r["status"] == "SKIP" + assert "environment: provider key out of credit" in r["why"] + + +def test_c5_keeps_fail_for_an_unrelated_error(monkeypatch): + turn = _Turn(errors=["sandbox create timed out after 120s"]) + mod = _c5(monkeypatch, turn) + r = mod.c5_first_call_race() + assert r["status"] == "FAIL" + + +def test_c5_still_fails_the_incident_even_when_the_copy_names_credits(monkeypatch): + """The incident is more specific than a credits failure and must win. + + Add-a-key copy over a placeholder refusal is the production bug this whole cell exists for. + The out-of-credit SKIP must not swallow it just because the wording mentions credits. + """ + turn = _Turn( + errors=["LiteLLM Virtual Key expected. Received=dtn_secret_abc"], + frames=[ + { + "type": "data-agent-error", + "data": { + "code": "starter_credits_exhausted", + "errorText": "Your free Agenta credits are used up. Add your own provider key to keep going.", + }, + } + ], + ) + mod = _c5(monkeypatch, turn) + r = mod.c5_first_call_race() + assert r["status"] == "FAIL" + assert "reported as the user's key problem" in r["why"] + + +def _teardown(monkeypatch, turn): + monkeypatch.setenv("AGENTA_BASE", "http://localhost:9999") + monkeypatch.setenv("AGENTA_PROJECT_ID", "proj-1") + monkeypatch.setenv("AGENTA_API_KEY", "test-key") + monkeypatch.setenv("DAYTONA_API_KEY", "test-daytona-key") + sys.path.insert(0, str(Path(__file__).resolve().parent)) + sys.modules.pop("qa_matrix_lib", None) + sys.modules.pop("check_secrets_teardown", None) + mod = importlib.import_module("check_secrets_teardown") + monkeypatch.setattr(mod, "list_secret_ids_by_name", lambda *a, **k: {}) + monkeypatch.setattr(mod, "create_workflow", lambda *a, **k: ("wf-1", "var-1")) + monkeypatch.setattr(mod, "seed_and_baseline", lambda *a, **k: ("rev-1", 1)) + monkeypatch.setattr(mod, "refs", lambda *a, **k: {}) + monkeypatch.setattr(mod, "archive", lambda *a, **k: None) + monkeypatch.setattr(mod, "invoke", lambda *a, **k: turn) + return mod + + +def test_teardown_skips_an_exhausted_key(monkeypatch): + turn = _Turn( + errors=["Your free Agenta credits are used up. Add your own provider key."] + ) + mod = _teardown(monkeypatch, turn) + with pytest.raises(mod.SkipCheck) as e: + mod.secrets_teardown(settle_seconds=1) + assert "environment: provider key out of credit" in str(e.value) + + +def test_teardown_keeps_fail_for_an_unrelated_error(monkeypatch): + turn = _Turn(errors=["sandbox create timed out after 120s"]) + mod = _teardown(monkeypatch, turn) + r = mod.secrets_teardown(settle_seconds=1) + assert r["status"] == "FAIL" + assert "the journey never ran" in r["why"] diff --git a/.agents/skills/agent-release-gate/resources/test_sweep_disagree_exceptions.py b/.agents/skills/agent-release-gate/resources/test_sweep_disagree_exceptions.py new file mode 100644 index 0000000000..56d4ede980 --- /dev/null +++ b/.agents/skills/agent-release-gate/resources/test_sweep_disagree_exceptions.py @@ -0,0 +1,159 @@ +"""Unit test for the sweep's triaged comparator-gap exceptions (run: `pytest +test_sweep_disagree_exceptions.py`). + +The tension this pins: `sweep_disagree.py` exists to fail on identity drift, and every exception +added to it is a place a real regression could hide. The 2026-08-31 triage found all 9 DISAGREE +lines were shadow-comparator modeling gaps — the coordinator is correct and pinned; only the +shadow's model of it disagrees — so without exceptions the sweep fails on the runner's own +expected behavior on every loaded window, and a check that cries wolf stops being read. + +So the exceptions have to be narrow in BOTH directions, and that is what these tests hold: + + - each triaged shape is excluded, and the excluded line is still PRINTED with its shape name + and the triage marker (an exception a reader cannot trace is indistinguishable from a bug + being hidden); + - a DISAGREE line matching no triaged shape still FAILS; + - a mixed window reports both counts, so an exclusion can never mask a real hit; + - each shape is anchored on BOTH halves of the line, so it cannot swallow a real disagreement + that merely shares a decision reason. + +The lines below are the real shapes from the triage, in the exact format `logReconcileShadow` +writes. +""" + +import importlib +import sys +from pathlib import Path + +import pytest + + +def _sweep(): + sys.path.insert(0, str(Path(__file__).resolve().parent)) + sys.modules.pop("sweep_disagree", None) + return importlib.import_module("sweep_disagree") + + +def line(decision: str, plan: str, facets: str = "harnessSession") -> str: + return ( + f"[reconcile] shadow key=proj:sess harness=claude decision={decision} " + f"plan={plan} DISAGREE facets=[{facets}]" + ) + + +# The three triaged shapes, as they really appear. +CLUSTER_7X = line("rebuild(mismatch:config)", "reuse(reopen-session)") +APPROVAL_MISMATCH = line("rebuild(approval-mismatch:unknown)", "reuse(no-op)", "none") +APPROVAL_RESUME = line( + "reuse(approval-resume)", "rebuild(rebuild-sandbox)", "workspaceFiles" +) + +# Something no triage covers: this must always fail. +NOVEL = line("rebuild(mismatch:credentials)", "reuse(no-op)", "modelCredential") + + +@pytest.mark.parametrize( + "raw,expected_shape", + [ + (CLUSTER_7X, "reopen-session-vs-config-rebuild"), + (APPROVAL_MISMATCH, "approval-mismatch-under-environment-scope"), + (APPROVAL_RESUME, "approval-resume-deferral"), + ], +) +def test_each_triaged_shape_is_recognized(raw, expected_shape): + m = _sweep() + shape = m.classify_disagree(raw) + assert shape is not None + assert shape[0] == expected_shape + assert shape[1], "every shape must carry a why, for the printed line" + + +@pytest.mark.parametrize("raw", [CLUSTER_7X, APPROVAL_MISMATCH, APPROVAL_RESUME]) +def test_a_triaged_shape_is_excluded_but_still_carried(raw): + """Excluded, never dropped: the line comes back so the caller can print it.""" + m = _sweep() + excluded, unexplained = m.partition_known_gaps([raw]) + assert unexplained == [] + assert len(excluded) == 1 + name, why, carried = excluded[0] + assert carried == raw + assert name and why + + +def test_an_unlisted_shape_still_fails(): + m = _sweep() + excluded, unexplained = m.partition_known_gaps([NOVEL]) + assert excluded == [] + assert unexplained == [NOVEL] + + +def test_a_mixed_window_reports_both_counts(): + m = _sweep() + window = [CLUSTER_7X, NOVEL, APPROVAL_RESUME, CLUSTER_7X] + excluded, unexplained = m.partition_known_gaps(window) + assert len(excluded) == 3 + assert unexplained == [NOVEL] + + +def test_no_exceptions_fails_on_everything(): + """The flag that proves a shape can be deleted once its comparator fix lands.""" + m = _sweep() + window = [CLUSTER_7X, APPROVAL_MISMATCH, APPROVAL_RESUME] + excluded, unexplained = m.partition_known_gaps(window, use_exceptions=False) + assert excluded == [] + assert unexplained == window + + +class TestShapesAreAnchoredOnBothHalves: + """A shape keyed on the decision alone would swallow real disagreements sharing a reason.""" + + def test_config_rebuild_against_a_different_plan_is_not_excluded(self): + m = _sweep() + # Same decision as the 7x cluster, but the plan wanted a full rebuild-sandbox. That is a + # genuine disagreement about the ROUTE and must not be waved through. + assert ( + m.classify_disagree( + line("rebuild(mismatch:config)", "rebuild(rebuild-sandbox)") + ) + is None + ) + + def test_reopen_plan_against_a_different_decision_is_not_excluded(self): + m = _sweep() + assert ( + m.classify_disagree(line("reuse(hit-continue)", "reuse(reopen-session)")) + is None + ) + + def test_a_different_approval_mismatch_reason_is_not_excluded(self): + m = _sweep() + assert ( + m.classify_disagree( + line("rebuild(approval-mismatch:stale)", "reuse(no-op)") + ) + is None + ) + + def test_approval_resume_against_a_reuse_plan_is_not_excluded(self): + m = _sweep() + # The triaged shape is the resume deferral, where the plan wanted a REBUILD. An + # approval-resume that disagrees toward reuse is something else entirely. + assert ( + m.classify_disagree(line("reuse(approval-resume)", "reuse(no-op)")) is None + ) + + +def test_the_triage_marker_names_its_source(): + # The provenance travels with every excluded line; a bare "known issue" would be unfalsifiable. + m = _sweep() + assert "f7-disagree-triage.md" in m.TRIAGE_MARKER + assert "2026-08-31" in m.TRIAGE_MARKER + + +def test_an_agree_line_is_never_a_hit_in_the_first_place(): + m = _sweep() + agree = ( + "[reconcile] shadow key=proj:sess harness=claude decision=reuse(hit-continue) " + "plan=reuse(no-op) agree facets=[]" + ) + assert not m.DISAGREE_LINE.search(agree) diff --git a/.agents/skills/gitbutler-stacks/SKILL.md b/.agents/skills/gitbutler-stacks/SKILL.md new file mode 100644 index 0000000000..e111c379f0 --- /dev/null +++ b/.agents/skills/gitbutler-stacks/SKILL.md @@ -0,0 +1,176 @@ +--- +name: gitbutler-stacks +description: Hard-won GitButler mechanics for multi-lane work in this repo — committing to a specific lane in a stack, spreading a pile of edits back across an existing stack, ordering a stack and setting PR bases, and recovering from a scrambled workspace. Use when working with stacked branches, when `but rub`/`but absorb`/`but commit --only` mis-routes a change, when a stack collapses or a commit lands on the wrong lane, or when a hunk gets dropped. Not needed for ordinary single-lane work. +--- + +# GitButler stacks (Agenta) + +Everyday `but` usage — `but status`, `but commit`, `but push`, `but absorb` — is covered by +the root `AGENTS.md`. This skill is the multi-lane layer: it only matters once you have a +**stack** of branches, which is roughly 6% of the work here. + +**Read the first rule first.** Most of what follows exists to undo damage that only happens +when edits are made first and assigned to lanes afterward. Committing each change to its lane +as you go avoids nearly all of it. + +## Prefer one lane at a time + +Land a lane before starting the next. A single lane per session needs none of the machinery +below — no `--only` staging discipline, no stash isolation, no oplog restores. Reach for a +stack only when a change genuinely depends on another in-flight branch's commits. + +Sync a lane with **rebase, not merge**. Merge commits between branches are what collapse a +series (see the first hard-won gotcha). + +## Committing to specific lanes in a stack (the part that bites) + +Changes are assigned to the **stack**, not to an individual branch. `but rub +` and `but commit --only` both operate on the stack's *assigned-changes* +set — `--only` commits **whatever is currently assigned** to the named branch, regardless +of which branch name you used when staging. So: + +- **Never pre-stage multiple lanes' files and then commit them one lane at a time.** The + first `but commit --only` sweeps the entire assigned set into that one branch (the others + end up empty or scrambled). Instead, work **one lane at a time**: assign exactly that + lane's files → `but commit --only` → **verify** → then assign the next lane's + files. Keep the assigned set equal to exactly one lane's files at each commit. +- **Verify every commit immediately:** `git show --stat --name-only `. If a file + from another lane leaked in, stop and fix before continuing. +- **`but rub` by path goes stale after any mutation.** Every `but` mutation kicks a + background sync that invalidates the path index, so the *next* path-based + `but rub ...` often fails with "Source '' not found". Use the stable + **cliId** instead (the 2-4 char code in `but status` / `but status --json`): + `but rub `. cliIds survive across the sync; paths don't. +- **Splitting one file across two stacked lanes** (e.g. `routers.py` where the lower lane + owns half the edit and the upper lane the other half): you cannot split mixed hunks + reliably. Instead use sequential working-tree states — make the file the lower lane's + version, commit it to the lower lane; then edit the file to add the upper lane's delta + and `but rub ` to amend that delta into the upper commit. +- The **branch ref can diverge from the workspace-applied commit** mid-session (after + absorb/amend/rebase). The **working tree is the source of truth**; `but push` pushes the + applied state. Don't panic if `git diff -- ` shows a delta while + `git status` is clean — verify against `git show ":"` and re-push. + +## Spreading a pile of edits back across an existing stack (the reliable way) + +When you have a working tree full of changes that belong to *many* lanes of an +already-pushed stack (e.g. a review-pass that fixes files across wp0…wp4), do NOT try to +assign-and-commit lane by lane against the live working tree — `but rub`/`but commit +--only`/`but absorb` all route by **hunk dependency across the whole stack**, and they +mis-route in three predictable ways that scramble the stack and waste hours: + +- **New (untracked) files ignore the target branch.** `but rub ` + dumps every untracked file into the **topmost** lane's staging group, not the one you + named. New files cannot be assigned to a lower lane at all. +- **`but absorb` sends anything it can't attribute to the docs/top lane.** Renamed files, + new files, and hunks in line-regions the target lane's original commit never touched all + fall to the "last commit in the primary lane" fallback — silently the wrong lane. +- **A multi-hunk file whose hunks belong to different commits won't commit whole.** `but + commit ` / `-p ` commits the attributable hunks and **drops the rest** + ("Warning: Some selected changes could not be committed"), often leaving an empty + no-change commit. Splitting one file across lower+upper lanes is the §"Splitting one file + across two stacked lanes" case above. + +The technique that actually works — **git-stash isolation, one lane at a time:** + +1. `but oplog snapshot -m "pristine"` then `git stash push -u` everything. Working tree + clean, every lane back at its remote tip. This snapshot is your only safe recovery + point — `but oplog restore` it whenever a step scrambles the stack (it does, often). +2. For each lane, restore **only that lane's files** into the clean tree: + tracked-modified from `git checkout 'stash@{0}' -- `; **untracked/new** files + from the stash's untracked parent `git checkout 'stash@{0}^3' -- `; reproduce + deletes/renames with `git rm`. Verify with `git status` that ONLY that lane's files are + present — nothing else. +3. Land them: if every hunk dependency-attributes cleanly to existing commits in that lane + (and the lane below), a blanket `but absorb` (no source — the tree holds only this + lane's files, so there's nothing to mis-route) puts each hunk in the right commit. If + the lane needs **new** files, use `but commit ` instead (the new files have only + this lane to land in because the tree is isolated). +4. **Verify the lane's tip TREE, not the diff** (commit history within a lane doesn't + matter; the resulting tree does): `git show :` for each touched file, plus + `git ls-tree -r ` for moves/deletes. Then check the lanes *above* it for + resurrected deletes / phantom files (the rebase re-materializes deleted dirs as + untracked — `rm -rf` that residue; it's noise, the tip tree is authoritative). +5. Next lane. Push at the very end with `but push -f` and confirm every lane's + `git rev-parse ` == `git ls-remote origin `. + +Unrelated fixes that depend on nothing in the stack (e.g. a stale test for code already on +main) go on their **own parallel lane**: isolate just that file, `but commit -c `. + +## Stacks are linear; a fan-out is expressed through PR bases, not graph shape + +A GitButler **stack** is a linear series. `but branch new --anchor ` does NOT +create a sibling of `` — it **inserts the new branch into the line** on top of it. So +anchoring two branches on the same parent produces `parent → first → second`, not two children +of `parent`. `but branch new ` with **no** anchor makes a separate parallel stack, but a +parallel stack branches off the workspace base (main), so a branch that genuinely depends on an +ancestor's commits can't live there with a clean diff. + +This matters when a design's dependency tree fans out (e.g. a web lane and an SDK lane that both +depend on an API lane but not on each other). You cannot draw that fan-out in the git graph here. +You don't need to. The clean per-PR diff is a **PR-base** property, not a graph-shape property: +a stacked branch contains every commit below it, and GitHub shows only the delta against the base +you set. So put everything in **one linear stack in dependency order** and set each PR's base to +the branch directly below it. Order independent lanes however you like (sort by fewest conflicts); +lanes that touch disjoint files (e.g. `web/**` vs `api/**`) can sit anywhere in the line. + +- Build the line with `but move ` (stacks `` on top of ``) + and `but move zz` (tears `` off into its own parallel stack). Use these to + reorder after the fact; take a `but oplog snapshot` first. +- **Verify the line by diffing, not by eyeballing the tree.** For each branch, run + `git diff --name-only ..` where `` is the branch below it. The file list + must be exactly that lane's files. If a lower lane's files appear, the order is wrong (a lane got + inserted into another's ancestry) — `but move` it out of the way and re-diff. +- A branch torn off to its own parallel stack (base = main) gives a **wrong** diff against an + ancestor branch: `git diff ..` reverses the ancestor's own changes (their + merge base is main). That's the tell that the branch needs to be stacked, not parallel. +- Set PR bases to match: bottom lane `--base main`, every other lane `--base `. + +## Hard-won gotchas (don't relearn these) + +- **GitButler series need linear history.** A stack of branches connected by + `git merge` commits (e.g. branches synced by merging a release in) can collapse + to a single series (the tip) when unapplied/re-applied — the intermediate + branches stop being addressable and you can't `but commit` to them. Prefer + GitButler's own stacking over merging branches into each other. +- **Don't sync a behind lane with `unapply` → `git branch -f origin/` → + `apply`.** Pointing a series at a merge-based origin ref flattens the stack. + There is no clean "fast-forward this series to its own remote" in the CLI when + origin is merge-based and ahead. +- **`but pull` rebases applied branches on the TARGET (main), not on each + branch's own upstream.** It will not advance a series to `origin/`. +- **Recovery: `but oplog list` then `but oplog restore `** rewinds the whole + workspace (including uncommitted changes) to any prior snapshot — this is how + you undo a botched unapply/apply and get a collapsed stack's series back. Take + a `but oplog snapshot -m "..."` before risky operations. +- **A dropped-hunk commit can damage the working tree, not just the commit.** + When `but commit --only` warns "Some selected changes could not be committed", + check the FILE in the tree afterwards, not only the commit: the reconcile can + rewrite the working copy and silently lose the uncommitted hunks. Recover the + exact bytes from a snapshot's worktree subtree: + `git cat-file -p :worktree/` (every oplog entry stores one). +- **Hunks land on the lane whose commits own their line regions — put the file + there instead of fighting.** If a file's edits sit in regions a higher lane's + commit created, committing them to the lower lane drops them and `but absorb` + amends the higher lane's commit anyway (and an absorb that amends a LOWER + commit can leave a `{conflicted}` commit above it; if that happens, restore + the snapshot rather than uncommitting around it). Keep code and its tests + together on whichever lane attribution chooses. +- **Parse cliIds from `but status --json`** (`filePath`/`cliId` pairs), never by + grepping the human output — the graph art breaks naive extraction and a wrong + token silently rubs the wrong thing. Commit cliIds also rotate after every + background sync, so re-read them in the same breath as the command that uses + them. +- **Lane a test with the half that appears LAST, not the half you were thinking + about.** A test that drives two halves (a DAO and its wrapper, an API parser + and the SDK catalog) fails in isolation on every lane between them if it lands + below the upper half — and a green local run proves nothing, because the + working tree has all lanes applied. The cheap check before landing any new + test file: ask which lane's tip FIRST contains every symbol the test touches, + and put the test there. (Three instances in one day before this rule.) +- **Remote-tracking refs go stale under `but push` and lie convincingly.** + `git show /:` can show old content long after the push + landed (observed: the tracking ref pointing at neither the local ref nor the + real remote head). Verify pushes ONLY with `git ls-remote` against + `git rev-parse `, and inspect remote content via the commit object, + never via the remote-tracking shortcut. diff --git a/.claude/skills/gitbutler-stacks b/.claude/skills/gitbutler-stacks new file mode 120000 index 0000000000..b2bd5e8027 --- /dev/null +++ b/.claude/skills/gitbutler-stacks @@ -0,0 +1 @@ +../../.agents/skills/gitbutler-stacks \ No newline at end of file diff --git a/.gitignore b/.gitignore index 5f8c38050b..16932179ff 100644 --- a/.gitignore +++ b/.gitignore @@ -94,6 +94,7 @@ services/runner/tests/results/ !.agents/skills/agent-release-gate/ !.agents/skills/agenta-package-practices/ !.agents/skills/create-changelog-announcement/ +!.agents/skills/gitbutler-stacks/ !.agents/skills/implement-feature/ !.agents/skills/plan-feature/ !.agents/skills/style-editing/ @@ -135,6 +136,7 @@ services/runner/tests/results/ !.claude/skills/mobile-app-structure !.claude/skills/mobile-shadcn-conventions !.claude/skills/mobile-motion-patterns +!.claude/skills/gitbutler-stacks # Temporary SDK copies created by run.sh --local api/sdks diff --git a/.gitleaksignore b/.gitleaksignore index decbc8d39e..5c796507b5 100644 --- a/.gitleaksignore +++ b/.gitleaksignore @@ -301,3 +301,4 @@ ea3257c7cc43d635b7bd8a16863d26df8d012c85:api/oss/tests/pytest/unit/secrets/test_ 84bb7fe0c9f926c6a60d7bd8577ba64043b655e7:api/oss/tests/pytest/unit/secrets/test_managed_secrets.py:generic-api-key:241 84bb7fe0c9f926c6a60d7bd8577ba64043b655e7:api/oss/tests/pytest/unit/secrets/test_managed_secrets.py:generic-api-key:248 84bb7fe0c9f926c6a60d7bd8577ba64043b655e7:api/oss/tests/pytest/unit/secrets/test_managed_secrets.py:generic-api-key:303 +e30afd32abeb0b51f22a0590b5bf44899978712b:.agents/skills/agent-release-gate/resources/test_out_of_credit_skip.py:generic-api-key:50 diff --git a/AGENTS.md b/AGENTS.md index 9a102ec0d8..59e7b85e54 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -18,6 +18,7 @@ bottom. ## Where conventions live - Frontend (imports, state, data fetching, styling, React, Fern client): `web/AGENTS.md`. +- GitButler stacked branches (lane routing, recovery, PR bases): the `gitbutler-stacks` skill. - API architecture (layering, domains, endpoints, exceptions, DTOs): `api/AGENTS.md`. - Local dev stack run commands: `hosting/AGENTS.md`. - Package vs app placement, `@agenta/*` packages, package unit tests: the @@ -45,159 +46,17 @@ If so, use the `but` CLI instead of raw `git branch`/`git commit`: `git ls-remote --heads origin ` vs `git rev-parse `. They must match. - To update an already-committed file, `but absorb ` amends it into the right commit; force-push with `but push -f`. +- **Recovery:** `but oplog list` then `but oplog restore ` rewinds the whole workspace + (including uncommitted changes) to any prior snapshot. Take a `but oplog snapshot -m "..."` + before anything risky. -### Committing to specific lanes in a stack (the part that bites) +Sync a lane by **rebasing on main, not by merging main into it** — merge commits between +branches collapse a GitButler series. -Changes are assigned to the **stack**, not to an individual branch. `but rub -` and `but commit --only` both operate on the stack's *assigned-changes* -set — `--only` commits **whatever is currently assigned** to the named branch, regardless -of which branch name you used when staging. So: - -- **Never pre-stage multiple lanes' files and then commit them one lane at a time.** The - first `but commit --only` sweeps the entire assigned set into that one branch (the others - end up empty or scrambled). Instead, work **one lane at a time**: assign exactly that - lane's files → `but commit --only` → **verify** → then assign the next lane's - files. Keep the assigned set equal to exactly one lane's files at each commit. -- **Verify every commit immediately:** `git show --stat --name-only `. If a file - from another lane leaked in, stop and fix before continuing. -- **`but rub` by path goes stale after any mutation.** Every `but` mutation kicks a - background sync that invalidates the path index, so the *next* path-based - `but rub ...` often fails with "Source '' not found". Use the stable - **cliId** instead (the 2-4 char code in `but status` / `but status --json`): - `but rub `. cliIds survive across the sync; paths don't. -- **Splitting one file across two stacked lanes** (e.g. `routers.py` where the lower lane - owns half the edit and the upper lane the other half): you cannot split mixed hunks - reliably. Instead use sequential working-tree states — make the file the lower lane's - version, commit it to the lower lane; then edit the file to add the upper lane's delta - and `but rub ` to amend that delta into the upper commit. -- The **branch ref can diverge from the workspace-applied commit** mid-session (after - absorb/amend/rebase). The **working tree is the source of truth**; `but push` pushes the - applied state. Don't panic if `git diff -- ` shows a delta while - `git status` is clean — verify against `git show ":"` and re-push. - -### Spreading a pile of edits back across an existing stack (the reliable way) - -When you have a working tree full of changes that belong to *many* lanes of an -already-pushed stack (e.g. a review-pass that fixes files across wp0…wp4), do NOT try to -assign-and-commit lane by lane against the live working tree — `but rub`/`but commit ---only`/`but absorb` all route by **hunk dependency across the whole stack**, and they -mis-route in three predictable ways that scramble the stack and waste hours: - -- **New (untracked) files ignore the target branch.** `but rub ` - dumps every untracked file into the **topmost** lane's staging group, not the one you - named. New files cannot be assigned to a lower lane at all. -- **`but absorb` sends anything it can't attribute to the docs/top lane.** Renamed files, - new files, and hunks in line-regions the target lane's original commit never touched all - fall to the "last commit in the primary lane" fallback — silently the wrong lane. -- **A multi-hunk file whose hunks belong to different commits won't commit whole.** `but - commit ` / `-p ` commits the attributable hunks and **drops the rest** - ("Warning: Some selected changes could not be committed"), often leaving an empty - no-change commit. Splitting one file across lower+upper lanes is the §"Splitting one file - across two stacked lanes" case above. - -The technique that actually works — **git-stash isolation, one lane at a time:** - -1. `but oplog snapshot -m "pristine"` then `git stash push -u` everything. Working tree - clean, every lane back at its remote tip. This snapshot is your only safe recovery - point — `but oplog restore` it whenever a step scrambles the stack (it does, often). -2. For each lane, restore **only that lane's files** into the clean tree: - tracked-modified from `git checkout 'stash@{0}' -- `; **untracked/new** files - from the stash's untracked parent `git checkout 'stash@{0}^3' -- `; reproduce - deletes/renames with `git rm`. Verify with `git status` that ONLY that lane's files are - present — nothing else. -3. Land them: if every hunk dependency-attributes cleanly to existing commits in that lane - (and the lane below), a blanket `but absorb` (no source — the tree holds only this - lane's files, so there's nothing to mis-route) puts each hunk in the right commit. If - the lane needs **new** files, use `but commit ` instead (the new files have only - this lane to land in because the tree is isolated). -4. **Verify the lane's tip TREE, not the diff** (commit history within a lane doesn't - matter; the resulting tree does): `git show :` for each touched file, plus - `git ls-tree -r ` for moves/deletes. Then check the lanes *above* it for - resurrected deletes / phantom files (the rebase re-materializes deleted dirs as - untracked — `rm -rf` that residue; it's noise, the tip tree is authoritative). -5. Next lane. Push at the very end with `but push -f` and confirm every lane's - `git rev-parse ` == `git ls-remote origin `. - -Unrelated fixes that depend on nothing in the stack (e.g. a stale test for code already on -main) go on their **own parallel lane**: isolate just that file, `but commit -c `. - -### Stacks are linear; a fan-out is expressed through PR bases, not graph shape - -A GitButler **stack** is a linear series. `but branch new --anchor ` does NOT -create a sibling of `` — it **inserts the new branch into the line** on top of it. So -anchoring two branches on the same parent produces `parent → first → second`, not two children -of `parent`. `but branch new ` with **no** anchor makes a separate parallel stack, but a -parallel stack branches off the workspace base (main), so a branch that genuinely depends on an -ancestor's commits can't live there with a clean diff. - -This matters when a design's dependency tree fans out (e.g. a web lane and an SDK lane that both -depend on an API lane but not on each other). You cannot draw that fan-out in the git graph here. -You don't need to. The clean per-PR diff is a **PR-base** property, not a graph-shape property: -a stacked branch contains every commit below it, and GitHub shows only the delta against the base -you set. So put everything in **one linear stack in dependency order** and set each PR's base to -the branch directly below it. Order independent lanes however you like (sort by fewest conflicts); -lanes that touch disjoint files (e.g. `web/**` vs `api/**`) can sit anywhere in the line. - -- Build the line with `but move ` (stacks `` on top of ``) - and `but move zz` (tears `` off into its own parallel stack). Use these to - reorder after the fact; take a `but oplog snapshot` first. -- **Verify the line by diffing, not by eyeballing the tree.** For each branch, run - `git diff --name-only ..` where `` is the branch below it. The file list - must be exactly that lane's files. If a lower lane's files appear, the order is wrong (a lane got - inserted into another's ancestry) — `but move` it out of the way and re-diff. -- A branch torn off to its own parallel stack (base = main) gives a **wrong** diff against an - ancestor branch: `git diff ..` reverses the ancestor's own changes (their - merge base is main). That's the tell that the branch needs to be stacked, not parallel. -- Set PR bases to match: bottom lane `--base main`, every other lane `--base `. - -### Hard-won gotchas (don't relearn these) - -- **GitButler series need linear history.** A stack of branches connected by - `git merge` commits (e.g. branches synced by merging a release in) can collapse - to a single series (the tip) when unapplied/re-applied — the intermediate - branches stop being addressable and you can't `but commit` to them. Prefer - GitButler's own stacking over merging branches into each other. -- **Don't sync a behind lane with `unapply` → `git branch -f origin/` → - `apply`.** Pointing a series at a merge-based origin ref flattens the stack. - There is no clean "fast-forward this series to its own remote" in the CLI when - origin is merge-based and ahead. -- **`but pull` rebases applied branches on the TARGET (main), not on each - branch's own upstream.** It will not advance a series to `origin/`. -- **Recovery: `but oplog list` then `but oplog restore `** rewinds the whole - workspace (including uncommitted changes) to any prior snapshot — this is how - you undo a botched unapply/apply and get a collapsed stack's series back. Take - a `but oplog snapshot -m "..."` before risky operations. -- **A dropped-hunk commit can damage the working tree, not just the commit.** - When `but commit --only` warns "Some selected changes could not be committed", - check the FILE in the tree afterwards, not only the commit: the reconcile can - rewrite the working copy and silently lose the uncommitted hunks. Recover the - exact bytes from a snapshot's worktree subtree: - `git cat-file -p :worktree/` (every oplog entry stores one). -- **Hunks land on the lane whose commits own their line regions — put the file - there instead of fighting.** If a file's edits sit in regions a higher lane's - commit created, committing them to the lower lane drops them and `but absorb` - amends the higher lane's commit anyway (and an absorb that amends a LOWER - commit can leave a `{conflicted}` commit above it; if that happens, restore - the snapshot rather than uncommitting around it). Keep code and its tests - together on whichever lane attribution chooses. -- **Parse cliIds from `but status --json`** (`filePath`/`cliId` pairs), never by - grepping the human output — the graph art breaks naive extraction and a wrong - token silently rubs the wrong thing. Commit cliIds also rotate after every - background sync, so re-read them in the same breath as the command that uses - them. -- **Lane a test with the half that appears LAST, not the half you were thinking - about.** A test that drives two halves (a DAO and its wrapper, an API parser - and the SDK catalog) fails in isolation on every lane between them if it lands - below the upper half — and a green local run proves nothing, because the - working tree has all lanes applied. The cheap check before landing any new - test file: ask which lane's tip FIRST contains every symbol the test touches, - and put the test there. (Three instances in one day before this rule.) -- **Remote-tracking refs go stale under `but push` and lie convincingly.** - `git show /:` can show old content long after the push - landed (observed: the tracking ref pointing at neither the local ref nor the - real remote head). Verify pushes ONLY with `git ls-remote` against - `git rev-parse `, and inspect remote content via the commit object, - never via the remote-tracking shortcut. +**Stacked branches have their own rules**, and they are the source of most `but` pain: +mis-routed hunks, dropped hunks, stale cliIds, collapsed series. Load the +`gitbutler-stacks` skill before doing any multi-lane work; do not improvise from these +basics. ## Before committing diff --git a/api/ee/src/core/starter_credits_bridge/service.py b/api/ee/src/core/starter_credits_bridge/service.py index e603e2810c..bc1228573e 100644 --- a/api/ee/src/core/starter_credits_bridge/service.py +++ b/api/ee/src/core/starter_credits_bridge/service.py @@ -248,7 +248,7 @@ async def _mint_key( for attempt in range(_MINT_ATTEMPTS): try: # The proxy caps what a key may ask for (`upperbound_key_generate_params`: - # max_budget 5, max_parallel_requests 2, rpm 30, tpm 200000, duration 90d) + # max_budget 5, max_parallel_requests 2, rpm 30, tpm 1000000, duration 90d) # and fills an OMITTED duration with its cap, so every funded key expires # after 90 days. Send no duration rather than a longer one. Raising # `grant_usd` or a per-key limit in the policy payload ABOVE a cap makes diff --git a/api/ee/src/core/starter_credits_bridge/types.py b/api/ee/src/core/starter_credits_bridge/types.py index 81d66598c4..eb30c44a6c 100644 --- a/api/ee/src/core/starter_credits_bridge/types.py +++ b/api/ee/src/core/starter_credits_bridge/types.py @@ -194,5 +194,5 @@ class MintedKey(BaseModel): "grant_usd": 5.0, "key_max_parallel_requests": 2, "key_rpm_limit": 30, - "key_tpm_limit": 200_000, + "key_tpm_limit": 1_000_000, } diff --git a/api/ee/tests/pytest/unit/test_starter_credits_bridge_seeding.py b/api/ee/tests/pytest/unit/test_starter_credits_bridge_seeding.py index bf101852c9..5dc8be3178 100644 --- a/api/ee/tests/pytest/unit/test_starter_credits_bridge_seeding.py +++ b/api/ee/tests/pytest/unit/test_starter_credits_bridge_seeding.py @@ -849,7 +849,7 @@ async def test_unconfigured_posthog_uses_the_development_policy(self): assert policy is not None assert policy.grant_usd == 5.0 assert policy.global_daily == 1000 - assert policy.key_tpm_limit == 200_000 + assert policy.key_tpm_limit == 1_000_000 assert policy.block_digit_locals is False # The built-in domain list still applies through the union. assert policy.is_freemail("gmail.com") is True diff --git a/api/oss/src/apis/fastapi/workflows/router.py b/api/oss/src/apis/fastapi/workflows/router.py index 32ab5cf95a..3e073b0fb1 100644 --- a/api/oss/src/apis/fastapi/workflows/router.py +++ b/api/oss/src/apis/fastapi/workflows/router.py @@ -24,6 +24,7 @@ WorkflowsService, SimpleWorkflowsService, ) +from oss.src.core.workflows.types import InvalidAgentHarnessError from oss.src.core.environments.service import ( EnvironmentsService, ) @@ -1645,6 +1646,10 @@ async def _commit_workflow_revision( ) except RevisionConflictError as e: raise HTTPException(status_code=409, detail=e.to_detail()) from e + except InvalidAgentHarnessError as e: + # 422, the same status every other "your change is not committable" answer on this + # route uses: the request is well-formed, the configuration in it is not. + raise HTTPException(status_code=422, detail=e.to_detail()) from e except ChangeSetError as e: raise HTTPException(status_code=422, detail=e.to_detail()) from e except NonEmbeddableWorkflowReferenceError as e: diff --git a/api/oss/src/core/tools/platform_handlers.py b/api/oss/src/core/tools/platform_handlers.py index da41a9dbd1..31d15c000b 100644 --- a/api/oss/src/core/tools/platform_handlers.py +++ b/api/oss/src/core/tools/platform_handlers.py @@ -776,7 +776,10 @@ async def handle_commit_revision( from oss.src.core.workflows.change_set import AGENT_COMMIT_SCOPE, ChangeSetError from oss.src.core.workflows.dtos import WorkflowRevisionCommit from oss.src.core.workflows.service import RevisionConflictError - from oss.src.core.workflows.types import StaticWorkflowSlug + from oss.src.core.workflows.types import ( + InvalidAgentHarnessError, + StaticWorkflowSlug, + ) if workflows_service is None: raise PlatformToolHandlerRefused("commit_revision is unavailable.") @@ -836,6 +839,8 @@ async def handle_commit_revision( ) except ChangeSetError as e: return PlatformHandlerResult.failure(AgentError(**e.to_detail())) + except InvalidAgentHarnessError as e: + return PlatformHandlerResult.failure(AgentError(**e.to_detail())) except RevisionConflictError as e: return PlatformHandlerResult.failure(AgentError(**e.to_detail())) except CommitLockTimeout as e: diff --git a/api/oss/src/core/workflows/service.py b/api/oss/src/core/workflows/service.py index 8e83495fe9..5b83462740 100644 --- a/api/oss/src/core/workflows/service.py +++ b/api/oss/src/core/workflows/service.py @@ -23,6 +23,7 @@ retrieve_interface, ) from agenta.sdk.engines.tracing.propagation import inject +from agenta.sdk.agents import HarnessKind, InvalidHarnessKindError from oss.src.core.git.interfaces import GitDAOInterface from oss.src.core.sessions.watch.interfaces import SessionsWatchPublisherInterface @@ -123,6 +124,7 @@ from oss.src.core.workflows.static_catalog import normalize_static_version from oss.src.core.workflows.dtos import WorkflowServiceDetachedResponse from oss.src.core.workflows.types import ( + InvalidAgentHarnessError, StaticWorkflowSlug, WorkflowServiceUrlMissing, WorkflowDetachedStartFailed, @@ -209,6 +211,34 @@ def to_detail(self) -> Dict[str, Any]: } +def _reject_unreadable_harness_kind(data: Optional[dict]) -> None: + """Refuse a commit whose agent template names a harness that does not exist. + + Deliberately narrow. It reads ONE field, and only when the commit actually carries it, so + a workflow that is not an agent (and an agent commit that leaves the harness alone) takes + exactly the path it took before. An absent or null kind means "use the default" and is + left to the runtime, which is the behaviour every existing config relies on. + """ + if not isinstance(data, dict): + return + parameters = data.get("parameters") + if not isinstance(parameters, dict): + return + agent = parameters.get("agent") + if not isinstance(agent, dict): + return + harness = agent.get("harness") + if not isinstance(harness, dict): + return + kind = harness.get("kind") + if kind is None or not str(kind).strip(): + return + try: + HarnessKind.coerce(kind) + except InvalidHarnessKindError as e: + raise InvalidAgentHarnessError(value=kind, message=e.message) from e + + def _validate_persisted_shape(data: dict) -> None: """The engine's final gate: the finished tree must be storable as it stands. @@ -2171,6 +2201,11 @@ async def commit_workflow_revision_checked( workflow_revision_commit=workflow_revision_commit, ) + # Checked on the CANDIDATE, so both commit forms are covered by one call: the delta arm + # has already merged its operations onto the head by here, and a full-data commit is + # the data as sent. An unrunnable agent config must not become a revision. + _reject_unreadable_harness_kind(candidate.data) + # The no-change answer belongs to the ordered-operations surface. With the flag off # the commit path stays exactly today's: a legacy delta or a full-data commit that # produces the stored configuration still creates a revision, because callers in diff --git a/api/oss/src/core/workflows/types.py b/api/oss/src/core/workflows/types.py index a45c342424..3c9fb9587f 100644 --- a/api/oss/src/core/workflows/types.py +++ b/api/oss/src/core/workflows/types.py @@ -5,7 +5,8 @@ never raise ``HTTPException`` directly. """ -from typing import Optional +from math import isfinite +from typing import Any, Dict, Optional # Reserved-slug detection is canonical in the SDK (it also drives is_static inference there). The # API re-exports it so every write path can reject a reserved slug and every read path can @@ -15,6 +16,7 @@ STATIC_SLUG_PREFIX, is_static_workflow_slug, ) +from agenta.sdk.agents import HarnessKind class WorkflowError(Exception): @@ -44,6 +46,66 @@ def __init__(self, slug: str, message: Optional[str] = None): ) +def _json_safe_echo(value: Any) -> Any: + """A value echoed back to the caller must survive JSON serialization. + + Python's json parser accepts the non-standard `NaN` and `Infinity` literals in a request + body, so a caller really can send one as a harness kind. Starlette serializes a response + with `allow_nan=False`, so echoing that float verbatim would raise inside the response and + turn this refusal into exactly the 500 it exists to replace. + """ + if isinstance(value, float) and not isfinite(value): + return repr(value) + if isinstance(value, (str, int, float)): + return value + return str(value) + + +class InvalidAgentHarnessError(Exception): + """The commit carries an agent configuration whose harness the runtime cannot read. + + A config with an unreadable ``harness.kind`` can never run, so storing it only moves the + failure somewhere less useful: the commit answered 200 and the invoke died on the enum's + bare ``ValueError`` as an unhandled 500 (finding F4). The write boundary is the outermost + place the caller can still be told which field is wrong, so it is refused here. + + Deliberately NOT a :class:`WorkflowError`. That base takes a positional message and exists + for the failures ``handle_workflow_exceptions`` translates one by one; this one carries a + value and an agent-actionable envelope, and the commit route maps it to 422 itself. Joining + the family would change nothing today and would put it in the path of any future broad + ``except WorkflowError``, which is a behavior change this move does not want to make. + """ + + code = "invalid_harness_kind" + + def __init__(self, *, value: Any, message: str) -> None: + super().__init__(message) + self.value = value + self.message = message + + def to_detail(self) -> Dict[str, Any]: + """The canonical agent-actionable envelope. See `api/AGENTS.md`. + + NOT retryable: the same bytes carry the same unreadable value forever. The caller has + a way forward, which is the `next_step`, so the allowed values travel in `details` + rather than only inside the message. + """ + return { + "code": self.code, + "message": self.message, + "retryable": False, + "next_step": ( + "Set agent.harness.kind to one of the allowed harnesses and send the " + "commit again." + ), + "details": { + "field": "parameters.agent.harness.kind", + "value": _json_safe_echo(self.value), + "allowed": sorted(kind.value for kind in HarnessKind), + }, + } + + class WorkflowServiceUrlMissing(WorkflowError): """Raised when a revision has no runnable service URL to invoke (batch or detached).""" diff --git a/api/oss/tests/pytest/unit/workflows/test_commit_endpoint.py b/api/oss/tests/pytest/unit/workflows/test_commit_endpoint.py index 8a6b48048a..59e10bce40 100644 --- a/api/oss/tests/pytest/unit/workflows/test_commit_endpoint.py +++ b/api/oss/tests/pytest/unit/workflows/test_commit_endpoint.py @@ -13,12 +13,19 @@ import pytest from fastapi import HTTPException +from starlette.responses import JSONResponse from oss.src.core.embeds.exceptions import NonEmbeddableWorkflowReferenceError from oss.src.core.git.types import CommitLockTimeout, VariantNotFound from oss.src.core.workflows.change_set import ChangeSetError, Reason -from oss.src.core.workflows.service import CommitOutcome, RevisionConflictError -from oss.src.core.workflows.types import StaticWorkflowSlug +from oss.src.core.workflows.service import ( + CommitOutcome, + RevisionConflictError, +) +from oss.src.core.workflows.types import ( + InvalidAgentHarnessError, + StaticWorkflowSlug, +) VARIANT_ID = uuid4() @@ -243,6 +250,58 @@ async def test_a_moved_head_answers_409_with_the_current_head( assert caught.value.detail["retryable"] is False assert caught.value.detail["next_step"] + async def test_an_unreadable_harness_answers_422_and_names_the_field( + self, router, allow_access + ): + # F4: this used to answer 200 and persist a config that could never run. The failure + # then surfaced on invoke as an unhandled 500 whose body was a Python repr, far from + # the request that caused it. + router.workflows_service.commit_workflow_revision_checked.side_effect = ( + InvalidAgentHarnessError( + value="not_a_real_harness", + message="invalid harness.kind (str) 'not_a_real_harness'", + ) + ) + + with pytest.raises(HTTPException) as caught: + await _commit(router) + + assert caught.value.status_code == 422 + assert caught.value.detail["code"] == "invalid_harness_kind" + # Not retryable: the same bytes carry the same unreadable value forever. The way + # forward is the next_step, and the values that exist travel in `details`. + assert caught.value.detail["retryable"] is False + assert caught.value.detail["next_step"] + assert ( + caught.value.detail["details"]["field"] == "parameters.agent.harness.kind" + ) + assert "pi_core" in caught.value.detail["details"]["allowed"] + + @pytest.mark.parametrize( + "kind,echoed", + [(float("nan"), "nan"), (float("inf"), "inf")], + ) + async def test_a_non_finite_kind_still_answers_422_and_not_a_500( + self, router, allow_access, kind, echoed + ): + # Python's json parser accepts the non-standard `NaN` and `Infinity` literals in a + # request body, and Starlette serializes with `allow_nan=False`. Echoing the float + # verbatim raised inside the response and turned this refusal back into a 500. + router.workflows_service.commit_workflow_revision_checked.side_effect = ( + InvalidAgentHarnessError( + value=kind, + message=f"invalid harness.kind (float) {kind!r}", + ) + ) + + with pytest.raises(HTTPException) as caught: + await _commit(router) + + assert caught.value.status_code == 422 + assert caught.value.detail["details"]["value"] == echoed + # What the client actually receives has to serialize. + JSONResponse(caught.value.detail) + async def test_a_change_set_refusal_answers_422(self, router, allow_access): router.workflows_service.commit_workflow_revision_checked.side_effect = ( ChangeSetError(Reason.INVALID_DELTA, "both forms") diff --git a/api/oss/tests/pytest/unit/workflows/test_commit_harness_validation.py b/api/oss/tests/pytest/unit/workflows/test_commit_harness_validation.py new file mode 100644 index 0000000000..777ccba0c2 --- /dev/null +++ b/api/oss/tests/pytest/unit/workflows/test_commit_harness_validation.py @@ -0,0 +1,144 @@ +"""F4: a commit must not be able to store an agent config the runtime cannot run. + +The gate's H1 cell committed `harness.kind` as `12345` and as `"not_a_real_harness"`. Both were +accepted with a 200. The config was then unrunnable forever, and the invoke that proved it died +on an unhandled 500 whose body was a Python repr, far from the request that caused it. + +The check is deliberately narrow: one field, read only when the commit actually carries it. The +cases below pin both halves of that — the values it refuses, and the far larger set of commits it +must leave completely alone, including every config that never names a harness. +""" + +from unittest.mock import AsyncMock +from uuid import uuid4 + +import pytest +from starlette.responses import JSONResponse + +from oss.src.core.workflows.dtos import WorkflowRevisionCommit +from oss.src.core.workflows.service import ( + WorkflowsService, + _reject_unreadable_harness_kind, +) +from oss.src.core.workflows.types import InvalidAgentHarnessError + + +def _data(kind): + return {"parameters": {"agent": {"harness": {"kind": kind}}}} + + +class TestValuesItRefuses: + @pytest.mark.parametrize("kind", [12345, "not_a_real_harness", 0, "pi", "claude "]) + def test_a_kind_the_runtime_cannot_read_is_refused(self, kind): + with pytest.raises(InvalidAgentHarnessError) as caught: + _reject_unreadable_harness_kind(_data(kind)) + + assert caught.value.code == "invalid_harness_kind" + + def test_the_envelope_names_the_field_the_value_and_what_is_allowed(self): + with pytest.raises(InvalidAgentHarnessError) as caught: + _reject_unreadable_harness_kind(_data("not_a_real_harness")) + + detail = caught.value.to_detail() + assert detail["code"] == "invalid_harness_kind" + assert detail["retryable"] is False + assert detail["next_step"] + assert detail["details"]["field"] == "parameters.agent.harness.kind" + assert detail["details"]["value"] == "not_a_real_harness" + assert set(detail["details"]["allowed"]) == {"pi_core", "claude", "codex"} + + +class TestTheEchoedValueSurvivesTheResponse: + """The refusal must not become the 500 it exists to replace. + + Python's json parser accepts the non-standard `NaN` and `Infinity` literals in a request + body, so a caller really can send one as a harness kind. Starlette serializes a response + with `allow_nan=False`, so echoing that float verbatim raised inside the response itself. + """ + + @pytest.mark.parametrize( + "kind,echoed", + [ + (float("nan"), "nan"), + (float("inf"), "inf"), + (float("-inf"), "-inf"), + ], + ) + def test_a_non_finite_float_is_echoed_as_text(self, kind, echoed): + with pytest.raises(InvalidAgentHarnessError) as caught: + _reject_unreadable_harness_kind(_data(kind)) + + detail = caught.value.to_detail() + assert detail["details"]["value"] == echoed + # The whole envelope has to survive the response, not just this field. + JSONResponse(detail) + + @pytest.mark.parametrize("kind", [12345, "not_a_real_harness", 0]) + def test_an_ordinary_value_is_still_echoed_as_itself(self, kind): + with pytest.raises(InvalidAgentHarnessError) as caught: + _reject_unreadable_harness_kind(_data(kind)) + + detail = caught.value.to_detail() + assert detail["details"]["value"] == kind + JSONResponse(detail) + + +class TestCommitsItMustNotTouch: + @pytest.mark.parametrize("kind", ["pi_core", "claude", "codex", "PI_CORE"]) + def test_a_readable_kind_passes(self, kind): + _reject_unreadable_harness_kind(_data(kind)) + + def test_a_legacy_pi_agenta_config_still_commits(self): + # The experiment is gone, but revisions saved while it existed still carry the value + # and must stay editable. The runtime maps it to plain Pi on read. + _reject_unreadable_harness_kind(_data("pi_agenta")) + + @pytest.mark.parametrize("kind", [None, "", " "]) + def test_an_absent_kind_means_the_default_and_is_not_a_refusal(self, kind): + _reject_unreadable_harness_kind(_data(kind)) + + @pytest.mark.parametrize( + "data", + [ + None, + {}, + {"parameters": None}, + {"parameters": {}}, + {"parameters": {"agent": None}}, + {"parameters": {"agent": {"instructions": "hi"}}}, + {"parameters": {"agent": {"harness": None}}}, + {"parameters": {"agent": {"harness": {}}}}, + # A workflow that is not an agent at all: the prompt/chat shape. + {"parameters": {"prompt": {"llm_config": {"model": "gpt-5.5"}}}}, + ], + ) + def test_a_commit_that_does_not_carry_the_field_is_untouched(self, data): + _reject_unreadable_harness_kind(data) + + +class TestNothingIsPersisted: + """The point of the boundary: the refusal happens BEFORE the write, not after it.""" + + @pytest.mark.asyncio + async def test_the_checked_commit_refuses_before_it_reaches_the_dao(self): + workflows_dao = AsyncMock() + # No head yet, so the base check has nothing to compare and the commit walks straight + # to the write it must not reach. + workflows_dao.fetch_revision.return_value = None + service = WorkflowsService(workflows_dao=workflows_dao) + + with pytest.raises(InvalidAgentHarnessError): + await service.commit_workflow_revision_checked( + project_id=uuid4(), + user_id=uuid4(), + workflow_revision_commit=WorkflowRevisionCommit( + slug="qa-h1-harness", + workflow_variant_id=uuid4(), + data={ + "uri": "agenta:builtin:agent:v0", + "parameters": {"agent": {"harness": {"kind": 12345}}}, + }, + ), + ) + + workflows_dao.commit_revision.assert_not_awaited() diff --git a/api/pyproject.toml b/api/pyproject.toml index 33124712bb..a11948e82a 100644 --- a/api/pyproject.toml +++ b/api/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "api" -version = "0.114.3" +version = "0.114.4" description = "Agenta API" requires-python = ">=3.11,<3.14" authors = [ diff --git a/api/uv.lock b/api/uv.lock index 841ef85699..c5ac1c1b8f 100644 --- a/api/uv.lock +++ b/api/uv.lock @@ -8,7 +8,7 @@ resolution-markers = [ [[package]] name = "agenta" -version = "0.114.3" +version = "0.114.4" source = { editable = "../sdks/python" } dependencies = [ { name = "agenta-client" }, @@ -72,7 +72,7 @@ dev = [ [[package]] name = "agenta-client" -version = "0.114.3" +version = "0.114.4" source = { editable = "../clients/python" } dependencies = [ { name = "httpx" }, @@ -276,7 +276,7 @@ wheels = [ [[package]] name = "api" -version = "0.114.3" +version = "0.114.4" source = { virtual = "." } dependencies = [ { name = "agenta" }, diff --git a/clients/python/pyproject.toml b/clients/python/pyproject.toml index d6025ebb91..bdd1988fe6 100644 --- a/clients/python/pyproject.toml +++ b/clients/python/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "agenta-client" -version = "0.114.3" +version = "0.114.4" description = "Fern-generated Python client for the Agenta API." requires-python = ">=3.11,<3.14" authors = [ diff --git a/clients/python/uv.lock b/clients/python/uv.lock index 2f37bc4cc3..60045addc2 100644 --- a/clients/python/uv.lock +++ b/clients/python/uv.lock @@ -4,7 +4,7 @@ requires-python = ">=3.11, <3.14" [[package]] name = "agenta-client" -version = "0.114.3" +version = "0.114.4" source = { editable = "." } dependencies = [ { name = "httpx" }, diff --git a/docs/design/agent-workflows/documentation/adapters/pi.md b/docs/design/agent-workflows/documentation/adapters/pi.md index b53b0fe9fd..0f87d27311 100644 --- a/docs/design/agent-workflows/documentation/adapters/pi.md +++ b/docs/design/agent-workflows/documentation/adapters/pi.md @@ -10,7 +10,7 @@ pages first. This page assumes the relay and the wire contract. ## How Pi runs Pi runs over ACP, through sandbox-agent (`engines/sandbox_agent.ts`). The harness value -`pi_core` (plain Pi) and `pi_agenta` (Pi with Agenta's forced opinion) both map to the `pi` +`pi_core` (and the legacy `pi_agenta` spelling of a removed experiment) both map to the `pi` ACP agent. This is the one engine the runner has. The sandbox-agent daemon starts the `pi-acp` adapter, which starts the `pi` CLI. diff --git a/docs/design/agent-workflows/documentation/agent-configuration.md b/docs/design/agent-workflows/documentation/agent-configuration.md index db4f6acd02..86d61d47ba 100644 --- a/docs/design/agent-workflows/documentation/agent-configuration.md +++ b/docs/design/agent-workflows/documentation/agent-configuration.md @@ -104,7 +104,7 @@ Its fields and defaults: | `model` | `str` | `"gpt-5.5"` | `x-parameter: grouped_choice`, plain string | | `tools` | `List[ToolConfig]` | empty list | typed discriminated union | | `mcp_servers` | `List[MCPServerConfig]` | empty list | typed | -| `harness` | `Literal["pi_core","claude","pi_agenta"]` | `"pi_core"` | enum | +| `harness` | `Literal["pi_core","claude","codex"]` | `"pi_core"` | enum | | `sandbox` | `Literal["local","daytona"]` | `"local"` | enum | | `runner.permissions.default` | `Literal["allow","ask","deny","allow_reads"]` | `"allow_reads"` | enum, four modes | @@ -218,8 +218,8 @@ Legend: (a) catalog/schema, (b) SDK neutral config, (c) runtime. | model / provider | yes, `model: str` | yes, `Optional[str]` | wired to the runner | Loose string. No `ModelRef`, no provider enum. There is no separate provider field. | | tools | yes, strict list | yes, lenient coercion | wired, resolved to builtin names + tool specs | Entries strict, list lenient. The shipped default template fills it with Pi's four default built-ins (`read`, `bash`, `edit`, `write`); see [Tools](tools.md). | | mcp_servers | yes, strict list | yes | wired, resolved to runner MCP servers | Strict per entry. Claude supports external HTTP servers; Pi refuses them until its bridge exists. | -| skills | yes, embed/inline list | yes | wired | Author-settable (`SkillConfig` inline or `@ag.embed` references). The playground build-kit overlay embeds one skill, the `build-an-agent` playbook; the `pi_agenta` harness additionally force-unions `getting-started`. See below. | -| persona | no | no | wired but forced only | Not a config field. The Agenta harness hardcodes an append-system preamble. See below. | +| skills | yes, embed/inline list | yes | wired | Author-settable (`SkillConfig` inline or `@ag.embed` references). The playground build-kit overlay embeds one skill, the `build-an-agent` playbook. See below. | +| persona | no | no | removed | Not a config field. It was the removed `pi_agenta` harness's hardcoded append-system preamble. See below. | | agents_md | yes, `agents_md: str` | yes, as `instructions` | wired to `agentsMd` | The schema names it `agents_md`. The neutral config names it `instructions`. | | harness | yes, enum | yes, on `AgentConfig` | wired, picks the harness class | Enum-enforced. The runtime validates via `make_harness`. | | sandbox | yes, enum | yes, on `AgentConfig` | wired to the backend, absent from `SessionConfig` | Backend concern, not agent identity. | @@ -227,14 +227,11 @@ Legend: (a) catalog/schema, (b) SDK neutral config, (c) runtime. ## Notable gaps and quirks -`persona` is not author config; it is a runtime injection of the Agenta harness only (a forced -append-system string). `skills` used to work the same way, but is author config now: inline -`SkillConfig` packages or `@ag.embed` references the backend inlines before the runner sees -them. Two platform skills still arrive without the author writing anything: the playground -build-kit overlay embeds the `build-an-agent` playbook, and the `pi_agenta` harness -force-unions the `getting-started` skill (`AGENTA_FORCED_SKILLS` in -`sdks/python/agenta/sdk/agents/adapters/agenta_builtins.py`). Each is delivered exactly once. -Pi (`pi_core`) and Claude harnesses get no forced skills or persona. +`persona` is gone with the removed `pi_agenta` harness (it was that harness's forced +append-system string). `skills` is author config: inline `SkillConfig` packages or +`@ag.embed` references the backend inlines before the runner sees them. One platform skill +still arrives without the author writing anything: the playground build-kit overlay embeds +the `build-an-agent` playbook. No harness forces skills or a persona. Per-harness divergence is real in other ways, but not in permission enforcement anymore: the permission policy is now enforced on both Claude and Pi. Builtin tool names are dropped for diff --git a/docs/design/agent-workflows/documentation/architecture.md b/docs/design/agent-workflows/documentation/architecture.md index e07a9d7d09..3816fce81b 100644 --- a/docs/design/agent-workflows/documentation/architecture.md +++ b/docs/design/agent-workflows/documentation/architecture.md @@ -13,8 +13,8 @@ The runtime keeps two run choices configurable as fields on `AgentConfig` (`sdks/python/agenta/sdk/agents/dtos.py`): - **Harness:** which agent runs. Supported values are `pi_core`, `claude`, and experimental - `pi_agenta`. Default `pi_core`. `pi_core` and `pi_agenta` both drive the `pi` ACP agent; - `pi_agenta` is Pi with Agenta's forced opinion. + Default `pi_core`, which drives the `pi` ACP agent. (`pi_agenta`, a removed experiment, + is still read as `pi_core` so old stored configs run.) - **Sandbox:** where the run happens. Supported values are `local` and `daytona`. Default `local`. @@ -66,11 +66,12 @@ and tool credentials and passes them only in the scoped `/run` payloads that nee The deployed handler always uses `SandboxAgentBackend`. `select_backend` in `services/oss/src/agent/app.py:49` constructs `SandboxAgentBackend` for every run, regardless -of harness. So `pi_core`, `claude`, and `pi_agenta` all run through the sandbox-agent daemon +of harness. So `pi_core`, `claude`, and `codex` all run through the sandbox-agent daemon over ACP. The runner has one engine, the sandbox-agent ACP path (`engines/sandbox_agent.ts`). The -`harness` field on the `/run` request selects the ACP agent: `pi_core` and `pi_agenta` both +`harness` field on the `/run` request selects the ACP agent: `pi_core` (and the legacy +`pi_agenta` spelling) both map to the `pi` ACP agent, `claude` maps to `claude`. There is no engine selector on the wire. A legacy in-process Pi engine and an `InProcessPiBackend` adapter existed during the POC; both were removed. @@ -82,7 +83,7 @@ The SDK runtime models engines as `Backend` adapters | Backend | Status | Harnesses | Sandbox support | Notes | | --- | --- | --- | --- | --- | -| `SandboxAgentBackend` | Implemented | `pi_core`, `claude`, `pi_agenta` | `local`, `daytona` | The deployed path and the only engine. Drives `engines/sandbox_agent.ts`: starts the sandbox-agent daemon and an ACP adapter. `supported_harnesses` is `{pi_core, claude, pi_agenta}` (`adapters/sandbox_agent.py:121`). | +| `SandboxAgentBackend` | Implemented | `pi_core`, `claude`, `codex` | `local`, `daytona` | The deployed path and the only engine. Drives `engines/sandbox_agent.ts`: starts the sandbox-agent daemon and an ACP adapter. `supported_harnesses` is `{pi_core, claude, codex}` (`adapters/sandbox_agent.py`). | | `LocalBackend` | Not implemented | Intended: `pi_core`, `claude` | Local machine | Public class exists; `create_sandbox` and `create_session` raise `NotImplementedError` (`adapters/local.py:34`). | ## Harnesses @@ -90,15 +91,14 @@ The SDK runtime models engines as `Backend` adapters The SDK runtime models agent-specific behavior as `Harness` adapters (`sdks/python/agenta/sdk/agents/adapters/harnesses.py`). The Python class names are unchanged; only the harness string values changed (`HarnessType.PI` is `"pi_core"`, `HarnessType.AGENTA` -is `"pi_agenta"`, `HarnessType.CLAUDE` is `"claude"`). +is removed, `HarnessType.CLAUDE` is `"claude"`). | Harness | Value | Status | Notes | | --- | --- | --- | --- | | `PiHarness` | `pi_core` | Implemented | Native Pi tools, Pi prompt overrides, Pi tracing extension. Drives the `pi` ACP agent. | | `ClaudeHarness` | `claude` | Implemented | MCP-delivered tools, permission policy, runner-built tracing. No Pi built-in tools. Drives the `claude` ACP agent. | -| `AgentaHarness` | `pi_agenta` | Experimental | Pi with forced tools, forced skills, a base AGENTS.md preamble, and a persona. Drives the `pi` ACP agent plus forced extras. Content is still placeholder. | -The `pi_agenta` harness runs on the sandbox-agent path. The runner treats it as the `pi` ACP +The removed `pi_agenta` value still reads as the `pi` ACP agent and layers the forced skills and prompt extras on top (`services/agent/src/engines/sandbox_agent/run-plan.ts`). The QA matrix verified it on sandbox-agent local and Daytona (`projects/qa/findings.md`, F-002). diff --git a/docs/design/agent-workflows/documentation/protocol.md b/docs/design/agent-workflows/documentation/protocol.md index 0a2914d02c..8a7f2e163f 100644 --- a/docs/design/agent-workflows/documentation/protocol.md +++ b/docs/design/agent-workflows/documentation/protocol.md @@ -119,6 +119,7 @@ Request fields include: | `sessionId` | External conversation id. The runtime is cold and receives history in `messages`. | | `agentsMd` | Instructions that become `AGENTS.md`. | | `systemPrompt`, `appendSystemPrompt` | Pi prompt overrides. The sandbox-agent engine writes `SYSTEM.md` / `APPEND_SYSTEM.md` into the per-run Pi agent dir, local and Daytona. | +| `gatewayGuidance` | The derived gateway-tools instruction section (`{text, carrier}`). The runner splices it into the named prompt surface (`appendSystemPrompt` or `agentsMd`) at environment build; it is deliberately excluded from the session fingerprint, so adding an integration never evicts a warm session and the names list (worded as examples) refreshes on the next build. | | `skills` | Resolved inline skill packages (full `SKILL.md` content, with `@ag.embed` references inlined server-side), declared in the agent config. All three harnesses wire them; the runner materializes each into a skill dir (`pi_core`/`pi_agenta` through Pi's agent-dir scope, Claude under project-local `.claude/skills`). Omitted when none are declared. | | `model` | Requested model id. Not honored on the Pi ACP path; pi-acp accepts only its default model (see Ground Truth). | | `messages` | Conversation history and current turn. | diff --git a/docs/design/agent-workflows/documentation/skills.md b/docs/design/agent-workflows/documentation/skills.md index 4692b814fe..7ed1c5b4e8 100644 --- a/docs/design/agent-workflows/documentation/skills.md +++ b/docs/design/agent-workflows/documentation/skills.md @@ -67,7 +67,7 @@ says. `agent-workflows-qa` (shared) defines the test matrix for the agent runtime. Its three axes are the environment (sandbox-agent local, sandbox-agent Daytona, and the local SDK), the -harness (`pi_core`, `pi_agenta`, `claude`), and the capability under test. "Test with daytona, +harness (`pi_core`, `claude`, `codex`), and the capability under test. "Test with daytona, local pi, and claude, on both the SDK and the UI" is exactly a walk across these cells. Each test forces a capability with a token the model cannot guess, so a pass proves the capability ran. diff --git a/docs/design/daytona-secret-propagation/README.md b/docs/design/daytona-secret-propagation/README.md new file mode 100644 index 0000000000..9f429beea8 --- /dev/null +++ b/docs/design/daytona-secret-propagation/README.md @@ -0,0 +1,102 @@ +# Daytona Secret propagation: the placeholder-401 incident and its instruments + +Working notes for the 2026-08-29 finding on EU cloud production: a fresh sandbox's first +model call sometimes carries the raw `dtn_secret_` placeholder instead of the real key, +because Daytona applies a new Secret's substitution rule asynchronously and gives no +completion signal. The model proxy refuses the placeholder with a 401, and the user used to +read "model authentication failed — add the project's OpenAI key", which was wrong on every +count. + +## The measured facts (72h window, EU cloud prod) + +- 7 placeholder 401s at the LiteLLM proxy (`Virtual Key expected. Received=dtn_****…`), + each matching a failed runner turn, across 5 organizations. +- Every one was the FIRST outbound model call of a freshly created sandbox, 10–24 seconds + after its Secrets were created. Substitution never failed mid-session. +- 4 of 7 had a destroy-plus-delete of the previous same-host Secret 14–61 seconds earlier + (the eviction ordering: destroy sandbox → delete its Secrets → allocate new → create). +- Successful cold starts sit in the same age range (6–22s), so the lag is stochastic, not a + fixed delay. Sibling measurement from 2026-08-09: value UPDATES on an existing Secret take + 15–18s to reach a running sandbox against the docs' "within seconds". + +## The two hypotheses + +- **H1, create lag**: the substitution rule for a new Secret + sandbox pair converges + per-node in Daytona's egress layer, and the first call sometimes lands on a node that has + not converged. +- **H2, delete interference**: deleting an older Secret for the same host while the new + one propagates widens the window. + +## Probe results (2026-08-30, production org, target eu) — CORRECTED + +The first probe run concluded substitution was host-dependent ("a never-used host never +substitutes"). That was a measurement artifact: Daytona's egress proxy also performs +RESPONSE SCRUBBING — real values in responses are rewritten back to placeholders before +they reach the sandbox — so an echo service shows `dtn_...` whether or not substitution +happened. Proven by sending the literal real value in the header: the echo still read +`dtn_secret_...`. The correct instrument is a provider whose error body echoes a MASKED +key (api.openai.com: "Incorrect API key provided: sk-probe*****"), which scrubbing does +not rewrite. + +With that instrument (20 fresh-Secret first-sandbox samples, production create shape, +target eu, 28 sandboxes total, all cleaned up): + +- **15 of 20 sandboxes substituted on their FIRST request**, +1.5s to +2.9s after Secret + creation — including brand-new hosts. Creation order and the delete-then-create eviction + ordering do not matter. +- **5 of 20 never substituted at all** (raw placeholder for the full 90-180s watched, from + the first request onward). The distribution is bimodal: no sample landed between 3s and + 180s. A twin sandbox created against the SAME Secret substituted at its first request + while the stuck one stayed raw; stop+start did not repair it. One stuck sandbox returned + an Envoy "upstream connect error or disconnect/reset before headers". + +**Mechanism read:** a per-sandbox registration failure at create time, with no +reconciliation — not a propagation delay. Production's "10-24s after creation" was merely +when the first call happened; "only first calls fail" is survivorship (the 401 kills the +run and the sandbox is rebuilt). Today's stuck rate (~25%) is far above the ~3% in the +production log window, so the rate varies or the eu fleet was degraded on 2026-08-30. + +Consequence for the runner: waiting does not help a stuck sandbox. The preflight must +REBUILD instead of waiting (see the instruments below). + +## The instruments + +- The stuck-sandbox rate was measured with a throwaway probe script, not tracked here: it + created sandboxes through the runner's own Daytona SDK calls and read the masked-echo + instrument (api.openai.com's 401 body, which names the credential it received). An echo + service cannot serve as the instrument — Daytona scrubs real credential values out of + responses, so an echo of a HEALTHY credential comes back looking like a placeholder. A + variant that deleted the previous Secret first tested the eviction ordering, which the + 2026-08-30 runs showed is NOT a factor. Each run costs about one sandbox-minute and needs + an environment holding the runner's Daytona credentials. +- The runner now logs `[daytona-secrets] allocated/deleted n=… hosts=[…] ms=…` (counts, + hosts, and timing only — never ids, names, placeholders, or values), so future incidents + carry their own create/delete timeline instead of needing it reconstructed from eviction + lines. +- A placeholder-shaped 401 classifies as `credential_delivery_failed` + (`services/runner/src/engines/sandbox_agent/errors.ts`), with retry-flavored user copy. + +## Daytona's answer (2026-08-31, their support; a fix PR is in progress on their side) + +They confirmed the reproduction and gave operating guidance: + +- "Substitution is not guaranteed on the first call. Some sandboxes never get wired; + restart does not fix those. A new sandbox on the same Secret does." +- "If the provider logs dtn_secret_…, throw that sandbox away and create a new one. + Retrying or restarting the same one will not help." +- "If a retry on the same sandbox starts working within ~30s, you can keep it. If the + placeholder is still going out after that, recreate." (We deliberately convict at 10s, + below their bound: every healthy sandbox we measured answered on its first probe, their + fix is imminent, and waiting 30s only holds a stuck user turn. Product-owner decision, + 2026-08-31.) +- "Keep one long-lived Secret; you don't need a new Secret per run." (DECLINED for + compliance, and told to them: we hold users' secrets for the sandboxes and must delete + them as soon as the sandbox is thrown away. A TTL-based garbage collector would be a big + lifecycle change, and their fix is imminent, so we keep per-run Secrets.) +- They marked the wiring failure as priority and will report when their fix lands. + +## The guard that does not wait for Daytona + +The preflight (#6370): probe the fresh sandbox concurrently with acquire; convict as +stuck when the raw placeholder still echoes at the 10s grace; destroy and rebuild once. Tracked in the session todo list beside this +workspace. diff --git a/docs/design/lifecycle-cold-warm-audit/README.md b/docs/design/lifecycle-cold-warm-audit/README.md new file mode 100644 index 0000000000..11ba922f42 --- /dev/null +++ b/docs/design/lifecycle-cold-warm-audit/README.md @@ -0,0 +1,74 @@ +# Cold/warm lifecycle audit — 2026-08-29 + +A fresh-context audit of the session lifecycle code, commissioned after the harnessKind +wire-spelling bug (#6364) to find more of its class. Method: read +`reconciliation-router.ts`, `desired-state.ts`, `session-coordinator.ts`, +`session-identity.ts`, `session-pool.ts`, `engine.ts`, `run-plan.ts` and their tests; +cross-check every wire literal against `protocol.ts` and the Python SDK; confirm each +behavioral claim with a throwaway probe built on the `lifecycle-live-routes.test.ts` +harness. + +## Findings, most severe first + +1. **FIXED (#6372). The live route cleared every mismatch reason.** The coordinator's + else-if chain found only the FIRST reason, and a successful live model apply set + `mismatch = undefined` wholesale. A model switch riding an edited transcript, a rotated + credential, an expiring mount lease, or a stale tail continued warm past the skipped + checks (all four proven warm with the probe). Fixed by re-asking the ordered checks + after each repair; four pinned tests. + + Review found a second half of the same finding: re-asking returned the FIRST unresolved + reason, and that reason alone chose the teardown. `history` sorts ahead of the credential + checks, so a model switch carrying both an edited transcript and an undeliverable + rotation evicted as `history`, mapped to `continuity-invalid`, and PARKED a sandbox whose + daemon still held the old key. The eviction is now NAMED by the first reason and DISPOSED + by all of them, strictest wins; pinned with the three-way combination. + +2. **`modelCapabilities` defeats the live model route across modality changes.** It is + per-turn data (read only by the attachment-delivery chain) but sits in the fingerprint + and the `harnessSession` facet, and it CHANGES WITH THE MODEL (resolved input + modalities). Switching between a vision model and a text-only model moves two facets, + the mixed plan rebuilds, and the one live route works only when the two models agree on + modalities. Fix: move it to the per-turn-volatile list (the `workflowRevision` / + `isDraft` precedent), pinned in `lifecycle-desired-state.test.ts`. + +3. **FIXED (#6399, with 6). `harnessMode` is normalized in the fingerprint but raw in the facet.** The two views + can disagree (measured: fingerprint equal while `harnessSession` moves), which poisons + later plans into rebuilding. Reachable today only through a direct runner caller. + Fix: normalize in one place; add the reverse-direction "no input drift" assertion (a + change that moves a facet must move the fingerprint). + +4. **FIXED (#6398). The agent-mount artifact id is in no fingerprint and no facet (under-eviction).** + `runContext.workflow.artifact.id` signs the agent mount, sets its env var, and appends + its guidance — all baked at acquire — yet a changed or newly present id reuses the warm + sandbox (measured). Fix: add the id alone to the `sandbox` facet + fingerprint, with an + eviction test. + +5. **FIXED (#6400). `toolCallback.endpoint` is fingerprinted but consumed per-turn (over-eviction).** Low + reachability (stable per-deployment URL); same fix class as 2. + +6. **FIXED (#6399). `configFingerprint` matches `codex` on a bare wire literal inline.** Correct today, + but the exact shape of the #6364 bug (a literal plus a silent fall-through). Fix: + one exported harness normalizer used by `configFingerprint`, `run-plan.ts`, and + `reconciliation-router.ts`, with a round-trip test over the SDK enum. + +7. **FIXED (#6400). `applyReconcilePlan` treats every `apply-live` action as a model change** (ignores + `action.facet`). Unreachable today; becomes real the moment the credential plan routes + through the applier. Fix: switch on the facet, refuse the rest, one test. + +8. **DECLINED. `connection` `{mode, slug}` evicts on every harness but only Pi consumes it.** Very + low reachability; listed for completeness. Declined on review: design Decision 7 pins that a + custom provider identity change cold-starts on EVERY harness, and its test covers non-Pi. + The decline is recorded beside the field in `session-identity.ts` (#6400). + +Verified clean: the harness wire spellings after #6364 (both normalizers agree, with a +plan-build assertion), and the shadow-router's decision scoping. + +## Standing checks born from this + +- Gate: L1's model case is blocking on claude AND pi_core (#6371). +- New Relic: alert policy "Runner lifecycle audit" (policy 1731651) pages the operator + Slack webhook on any `DISAGREE` or `harness=unknown` runner log line. +- Unit: the repair-scoping pinned tests in `lifecycle-live-routes.test.ts` (#6372). + +Every finding is now fixed or declined; each fix landed with its pin (see the PR on each row). diff --git a/docs/design/playground-shortcut-discoverability/README.md b/docs/design/playground-shortcut-discoverability/README.md new file mode 100644 index 0000000000..869d90b918 --- /dev/null +++ b/docs/design/playground-shortcut-discoverability/README.md @@ -0,0 +1,27 @@ +# Playground shortcut discoverability + +The agent playground binds forty-three keyboard shortcuts across six files. Six of them name a +key on screen. The rest are unreachable unless you read the source. + +This project gives every shortcut a place where a user can find it, adds two that were missing, +and moves three Alt letters off keys that browsers claim on Windows and Linux. + +## The files here + +| File | What it holds | +| ---------------------------- | -------------------------------------------------------------------- | +| [context.md](context.md) | What shipped before this project, and what the user asked for | +| [research.md](research.md) | The full inventory, the browser key conflicts, and what Linear does | +| [decisions.md](decisions.md) | **Why each key is what it is. Read this before changing a binding.** | +| [plan.md](plan.md) | The slices and their acceptance checks | +| [status.md](status.md) | What is done, what is left | + +## The short version + +- One registry, `web/packages/agenta-shared/src/utils/shortcuts.ts`, owns every binding's keys + and label. Tooltips, keycaps and the shortcuts sheet all read it, so a label can never drift + away from the handler. +- Keys appear on the control that already does the job: a tooltip, or a keycap on the button. +- The eleven shortcuts that answer no control live in a sheet, opened with `?` or from a + keyboard button at the right edge of the playground top bar. +- The Alt letters avoid every browser menu key. A unit test fails if one is ever bound again. diff --git a/docs/design/playground-shortcut-discoverability/context.md b/docs/design/playground-shortcut-discoverability/context.md new file mode 100644 index 0000000000..2f51873635 --- /dev/null +++ b/docs/design/playground-shortcut-discoverability/context.md @@ -0,0 +1,48 @@ +# Context + +## The symptom + +The agent playground binds forty-three keyboard shortcuts. They live in six separate files: + +| File | What it binds | +| --------------------------------------------------------------------------------------- | --------------------------------- | +| `web/oss/src/components/AgentChatSlice/hooks/useSessionShortcuts.ts` | the session and panel Alt chords | +| `web/oss/src/components/AgentChatSlice/AgentConversation.tsx` | `Esc` to stop, `Alt+G` to approve | +| `web/packages/agenta-ui/src/RichChatInput/plugins/SubmitPlugin.tsx` | the composer's Enter behaviour | +| `web/oss/src/components/AgentChatSlice/components/SlashCommand/useRovingList.ts` | the command picker | +| `web/packages/agenta-chat/src/components/ApprovalCard.tsx`, `ConnectionDock.tsx` | approve, deny, connect | +| `web/packages/agenta-chat/src/components/ElicitationDock.tsx`, `hooks/usePushToTalk.ts` | the agent's forms, and dictation | + +Six of the forty-three named a key on screen: the composer's send and newline hints, three in the +elicitation dock, and the voice button's hold label. The other thirty-seven were invisible, and a +user could only find them by reading the source. + +This project adds two more, the files pane and the shortcuts sheet itself, so the registry lists +forty-five. Every count in this workspace uses those two numbers: forty-three already shipped, +forty-five in the registry. + +Eleven of them answer no control at all. `Alt+1…9` and the `Alt+Z` / `Alt+X` pair switch +sessions, and there is no button anywhere to hang a tooltip on, so a tooltip pass alone could +never make them discoverable. + +## What the user asked for + +1. Find every shortcut and list them. +2. Propose where each one becomes visible. +3. Show the proposal in Storybook so it can be judged by eye before anything ships. +4. Confirm the shortcuts work on Windows, Linux and macOS, and change them if they do not. + +## Decisions the user made + +- The approval card shows its keys as keycaps inside the Approve and Deny buttons. Two other + variants were built and rejected: a hint line under the buttons, and tooltips only. +- No first-run nudge on the session strip. Shortcuts appear on their controls and in the + sheet, nowhere else. +- The shortcuts button sits at the right edge of the playground top bar, not in the session + bar. +- The sheet must fit a 15 inch screen without scrolling. + +## Related + +- `docs/design/agents-md-compartmentalization/playbook.md` for where instructions live. +- The `agenta-package-practices` skill for the package placement rules this work follows. diff --git a/docs/design/playground-shortcut-discoverability/decisions.md b/docs/design/playground-shortcut-discoverability/decisions.md new file mode 100644 index 0000000000..ee0415d80d --- /dev/null +++ b/docs/design/playground-shortcut-discoverability/decisions.md @@ -0,0 +1,96 @@ +# Key assignment decisions + +**Read this before you change, "simplify", or revert any Alt binding in the playground.** + +Several of these letters look arbitrary. They are not. Each one was moved off a key that a +browser already claims, and moving it back re-breaks the shortcut on Windows or Linux. + +The guard is `web/packages/agenta-ui/tests/unit/useSessionShortcuts.render.test.ts`, in the test +named "binds no letter a browser menu already claims". If you change a binding and that +test fails, the test is right and the change is wrong. + +## The letters a browser owns + +A web page that binds one of these fights the browser for the keystroke. Even where +`preventDefault` wins today, it is version-dependent and it is not worth relying on. + +| Chord | Who claims it | +| ------------------------------------------- | ------------------------------------------------------------------ | +| `Alt+F`, `Alt+E` | Chrome and Edge open their main menu. Firefox opens File and Edit. | +| `Alt+V`, `Alt+S`, `Alt+B`, `Alt+T`, `Alt+H` | Firefox opens View, History, Bookmarks, Tools and Help. | +| `Alt+D` | Chrome, Edge and Firefox all focus the address bar. | + +`Alt` plus a digit is claimed by nothing, on any of the three platforms. `Alt+1…9` is safe. + +## The letters macOS owns + +On a Mac, `Option` plus a letter usually types a character, which `preventDefault` suppresses. Five +are different: `Option+E`, `Option+I`, `Option+U`, `Option+N` and `Option+` are DEAD KEYS that begin +an accent. Binding one of them stops a user typing `é í ü ñ à` in the composer. + +This bit us. New session was moved to `Alt+N` on 2026-08-29, and `Option+N` is the tilde dead key, +so every macOS user would have created a session instead of typing `ñ`. It now uses the `+` key +(`event.code === "Equal"`), which matches the `+` button in the tab strip and is claimed by nothing. + +Never bind `Alt` plus E, I, U or N. + +## The letters are only proven safe in ENGLISH Firefox + +Firefox builds its menu access keys from the localised menu names, so the reserved set changes with +the interface language. English reserves F, E, V, S, B, T and H. German reserves D, B, A, C, L, X +and H, for Datei, Bearbeiten, Ansicht, Chronik, Lesezeichen, Extras and Hilfe. + +Three of our letters sit in the German set: `Alt+A` archives, `Alt+C` toggles the configuration and +`Alt+X` steps to the next session. A German Firefox user may get a menu instead. Other locales will +have their own sets, and we have not enumerated them. + +The unit test below proves only the English case. Do not read it as proof for any other language. +Whether to keep `Alt` plus a letter at all, or move to a three-key chord such as `Alt+Shift+letter` +the way Linear does, is an open decision recorded in [status.md](status.md). + +## What changed on 2026-08-29, and why + +| Action | Was | Now | Reason | +| ------------------- | ------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------- | +| Search sessions | `Alt+F` | `Alt+K` | `Alt+F` opens the browser menu. `K` is the search key across the industry, from `Cmd+K` command palettes. | +| Configuration panel | `Alt+B` | `Alt+C` | `Alt+B` opens Firefox's Bookmarks menu. `C` reads as Configuration. | +| New session | `Alt+C` | `Alt+N` | `N` reads as New, and moving it frees `C` for the panel above. | +| Files pane | none | `Alt+O` | It had no shortcut. `O` is free in every browser. It is the weakest mnemonic in the set; change it if a better free letter appears. | + +Unchanged, because they were already safe: `Alt+1…9`, `Alt+Z`, `Alt+X`, `Alt+W`, `Alt+R`, +`Alt+A`, `Alt+G`. + +## Why the Alt modifier stays at all + +Linear, the closest comparable product, binds plain single letters and two-letter runs such as +`g` then `i`, and reserves Alt for three-key combos only. That works because Linear's main +screens are lists and boards, where the caret is usually nowhere. + +The playground is the opposite. The caret sits in the composer nearly the whole time, so a +plain letter would be swallowed as typed text. A modifier is not a style choice here, it is a +requirement. `Cmd`/`Ctrl` plus a digit is browser tab switching on every operating system, and +`Cmd`/`Ctrl` plus most letters is already taken by the browser, so `Alt` is what is left. + +Three guards keep `Alt` from breaking normal typing, and all three are load-bearing: + +- The handler matches `event.code`, the physical key position, not the character. On macOS, + `Option` plus a letter types `ç Ω ≈ ∑ ® å ƒ ∫ ©`, and `Option+1` reports `event.key` as `¡`. +- The handler calls `preventDefault`, so none of those characters ever lands. +- The handler excludes `ctrlKey`. On European layouts AltGr reports as Ctrl plus Alt, so + excluding Ctrl keeps `@ { } [ ] €` typing normally. + +## Why `?` is matched on the character, not the position + +`?` is `Shift+/` on a US layout, `Shift+ß` on a German one and `Shift+,` on a French one. The +shortcuts sheet matches `event.key`, which reports the produced character, so it opens on all +three. It is the one binding in the playground that deliberately does not use `event.code`. + +It is also ignored whenever the caret is in an `input`, a `textarea`, or anything +content-editable, which includes the Lexical composer. Typing a question mark stays typing a +question mark. + +## Still open + +On Linux the window manager can claim an Alt chord before the browser ever sees it, and no +page code can help with that. Pressing all ten Alt chords once on a Linux desktop and once on +Windows would close this out. Nothing in the code can verify it. diff --git a/docs/design/playground-shortcut-discoverability/plan.md b/docs/design/playground-shortcut-discoverability/plan.md new file mode 100644 index 0000000000..e03580e71f --- /dev/null +++ b/docs/design/playground-shortcut-discoverability/plan.md @@ -0,0 +1,64 @@ +# Plan + +## Slices + +### S1 — the shortcut registry and the keycap primitive (done) + +One exported list owns every binding's keys and label, and one component prints them the way +the reader's own keyboard is labelled. + +- `web/packages/agenta-shared/src/utils/shortcuts.ts` — the registry, pure data, no React. +- `web/packages/agenta-ui/src/shortcuts/ShortcutKeys.tsx` — the keycaps. + +**Acceptance:** every hint in the app reads its keys from the registry. No component spells a +key string by hand. + +### S2 — the shortcuts sheet and its button (done, refined in S5) + +- `web/packages/agenta-ui/src/shortcuts/KeyboardShortcutsSheet.tsx` — the sheet plus the `?` + hotkey. +- `web/packages/agenta-ui/src/shortcuts/ShortcutsHelpButton.tsx` — the visible way in. + +**Acceptance:** `?` opens the sheet from the page and does nothing while the caret is in a +text field, where it types a question mark instead. + +### S3 — keys on the controls (done) + +- The approval card's Approve and Deny buttons carry keycaps. +- Both side panel carets name their key in their tooltip. + +**Acceptance:** hovering either caret shows its key; the approval card shows both keys without +a hover. + +### S4 — Windows and Linux safety (done) + +Move every Alt letter off the keys a browser menu claims. Full reasoning in +[decisions.md](decisions.md). + +**Acceptance:** the unit test "binds no letter a browser menu already claims" passes, and +every moved key is reflected in the registry, the handler, and the tooltips at once. + +### S5 — the button's home and the sheet's width + +- Move `ShortcutsHelpButton` out of the session bar and into the playground top bar, at the + right edge, after the settings gear. +- Widen the sheet and give it a third column on wide screens, so the full list fits a 15 inch + laptop without scrolling. + +**Acceptance:** the button is the rightmost control in the top bar. The sheet's content height +fits inside a 900px viewport with no scrollbar. + +### S6 — Storybook and the written record + +- The Storybook stories show only what shipped: one approval card, the sheet with its button, + and the placements. +- This workspace records why each key is what it is, so the next agent does not revert it. + +**Acceptance:** the Storybook builds clean, every story renders with no page errors, and +`decisions.md` explains every letter. + +## Out of scope + +- The remaining tooltips (the session tab menu's key column, the search box placeholder, the + stop button, the composer's `Shift+Enter` chip). They are listed in `status.md` as follow-up. +- Making the sheet searchable, as Linear's is. diff --git a/docs/design/playground-shortcut-discoverability/research.md b/docs/design/playground-shortcut-discoverability/research.md new file mode 100644 index 0000000000..98b8245eb9 --- /dev/null +++ b/docs/design/playground-shortcut-discoverability/research.md @@ -0,0 +1,119 @@ +# Research + +## The full inventory + +The registry `web/packages/agenta-shared/src/utils/shortcuts.ts` holds 45 entries. Forty-three +of them were already bound in the code before this project; two are new, the files pane and +the shortcuts sheet itself. Counts below are the registry's own groups. + +| Group | Entries | Named on screen before | +| -------------------- | ------- | ---------------------- | +| Sessions | 7 | 0 | +| Side panels | 2 | 0 (one did not exist) | +| While the agent runs | 2 | 0 | +| Composer | 6 | 2 | +| The `/` menu | 3 | 0 | +| Permission picker | 5 | 0 | +| Approval card | 2 | 0 | +| Connection dock | 2 | 0 | +| Forms the agent asks | 11 | 3 | +| Voice | 2 | 1 | +| Renaming a session | 2 | 0 | +| Help | 1 | 0 (did not exist) | + +The prompt playground's `Cmd/Ctrl+Enter` "Run all" is a separate page and is not in the +registry. It already names its key in a tooltip. + +### The `/` menu and the permission picker are two surfaces, not one + +They look like one list and they are not. Getting this wrong would have made the sheet lie. + +- The `/` menu is `web/packages/agenta-ui/src/RichChatInput/plugins/SlashCommandPlugin.tsx`. + It registers exactly five Lexical commands: ArrowDown, ArrowUp, Escape, Enter and Tab. Enter + and Tab both pick the active item. It binds no Home, no End, no ArrowLeft. +- The permission picker is `web/oss/src/components/AgentChatSlice/components/SlashCommand/useRovingList.ts`, + used only by `PermissionsPickerPanel.tsx`. That one binds Home, End and ArrowLeft. + +So the registry keeps them as separate groups. Merging them would promise Home and End inside +the `/` menu, where nothing answers. + +## What is safe on all three platforms + +Every `Cmd/Ctrl+Enter` binding tests `metaKey || ctrlKey`, so the same code answers Cmd on +Apple hardware and Ctrl everywhere else. Plain keys (`Esc`, the arrows, `Home`, `End`, +`Space`, the digits, `Enter`) have nothing platform-specific to collide with. + +Push to talk was already handled correctly before this project. It binds only the left Alt off +Apple hardware, because the right Alt is AltGr, and it arms only after a 300ms hold so a tap +types nothing. `web/packages/agenta-shared/src/utils/platform.ts` already exported +`modifierKeyLabel()`, `altKeyPrefix()` and `pushToTalkLabel()`, so the labels were +platform-aware before any of them were printed. + +## The browser key conflicts + +Measured against Chrome, Edge and Firefox on Windows and Linux. The table lives in +[decisions.md](decisions.md), which is the file to read before changing a binding. + +Two of the nine Alt letters sat on browser menu keys: search on `Alt+F` and the configuration +panel on `Alt+B`. Both are now moved. + +## Keystrokes leak through an open overlay + +Radix's dismissable layer +(`@radix-ui/react-dismissable-layer/dist/index.mjs:91-106`) listens for Escape in the capture +phase and calls `preventDefault()`, but it never calls `stopPropagation()`. Every bubble-phase +handler still runs. It does not touch `Cmd/Ctrl+Enter` at all. + +Adding a dialog the product tells you to open at any time made that reachable: pressing Escape +to close the shortcuts sheet also denied a parked tool call, and `Cmd/Ctrl+Enter` approved one +the user could not see. Opening the top bar's settings menu and pressing Escape denied a gate +the same way. + +Two guards are needed and each covers what the other misses: + +| Case | `isOverlayOpen()` | `event.defaultPrevented` | +| ------------------------------------ | ----------------- | ------------------------ | +| Escape under a dialog | catches | catches | +| `Cmd/Ctrl+Enter` under a dialog | catches | misses | +| Escape under a Radix menu or popover | misses | catches | +| Escape under an antd Modal | catches | misses | + +`isOverlayOpen()` misses menus and popovers because they are `role="menu"` and +`role="listbox"`, not `role="dialog"`. `defaultPrevented` misses the antd modal because +rc-dialog does not cancel the event, and misses `Cmd/Ctrl+Enter` because Radix never +intercepts it. + +One case stays deliberately uncovered. The workflow revision drawer is antd's `Drawer`, whose +panel renders `role="dialog"` with no `data-state`, so `isOverlayOpen()` returns false while it +is open. That is why the Alt shortcuts and `Esc`-to-stop already work inside that drawer, and +it must stay that way. + +## What Linear does + +Sources: + +- [Keyboard shortcuts help changelog](https://linear.app/changelog/2021-03-25-keyboard-shortcuts-help) +- [Linear keyboard shortcuts collection](https://keycombiner.com/collections/linear/) + +Findings: + +- `?` opens a searchable shortcuts panel. There is also a "Help & Feedback" entry in the + sidebar that reaches the same panel. A hotkey with no visible button teaches nobody, so the + two ship together. +- Navigation uses plain single letters and two-letter runs, such as `g` then `i` for the + inbox. Shortcuts go quiet while the user is editing. +- Global actions use `Ctrl`/`Cmd` plus a letter: `Ctrl+K` for the command menu, `Ctrl+I` to + open the details sidebar, `Ctrl+B` to switch list and board. +- Alt appears only in three-key combos, never as `Alt` plus a single letter. Examples: + `Alt+Shift+F` to clear filters, `Ctrl+Alt+1…9` to set a status. + +## Why we did not copy the plain-letter scheme + +Linear's plain letters work because its main screens are lists and boards, where the caret is +usually nowhere. The playground is the opposite: the caret sits in the composer nearly the +whole time. A plain letter there would be swallowed as typed text. + +`useSessionShortcuts.ts` states this constraint directly: the shortcuts must fire from any +focus context, the composer included, and that is the point of using a modifier. So the +modifier stays, and what we borrowed from Linear is the letter discipline, the `?` hotkey, and +the visible button beside it. diff --git a/docs/design/playground-shortcut-discoverability/status.md b/docs/design/playground-shortcut-discoverability/status.md new file mode 100644 index 0000000000..20dcdcd806 --- /dev/null +++ b/docs/design/playground-shortcut-discoverability/status.md @@ -0,0 +1,99 @@ +# Status + +**State:** ready for a PR. The GitButler lane is `feat/playground-shortcut-hints`. + +## Done + +- S1 the registry and the keycap primitive. +- S2 the shortcuts sheet and the `?` hotkey, with a visible button. +- S3 keycaps on the approval card, keys on both side panel carets. +- S4 the Alt letters moved off every browser menu key, with a test that keeps them off. + +- S5 the button sits at the right edge of the playground top bar, and the sheet is three + columns wide. +- S6 the Storybook pass and this written record. +- Keystrokes no longer leak through an open overlay. See the section in + [research.md](research.md); it was reachable the moment the sheet shipped. + +## Where the code lives, and why `/m` can reuse it + +Every piece sits in a package `/m` already depends on, so a mobile host wires callbacks rather +than reimplementing anything: + +| Piece | Home | +| --------------------------------------------------------------- | ---------------------- | +| The registry, the ARIA names, the overlay guard | `@agenta/shared/utils` | +| `useSessionShortcuts` | `@agenta/ui/shortcuts` | +| `ShortcutKeys`, `KeyboardShortcutsSheet`, `ShortcutsHelpButton` | `@agenta/ui/shortcuts` | + +`useSessionShortcuts` started in the app layer and moved here. It takes every action as a +callback and imports only React and `@agenta/shared/utils`, so nothing about it was +desktop-specific. A phone never sends an Alt chord, so mounting it on a touch surface is inert. + +What `/m` renders from this change today is the approval card with `touch` set, whose keycaps are +suppressed. The Storybook story **On mobile** pins that, because a keycap appearing there is a +regression nobody would catch by clicking on a desktop. + +The open product question, before any mobile wiring: which of these shortcuts belong on a surface +that is also used on a phone. Session switching and the panel toggles earn their place in a +desktop browser pointed at `/m`; on a handset they are dead weight. + +## Open decision: can `Alt` plus a letter work at all? + +Codex found that the "safe letters" list in [decisions.md](decisions.md) only holds for English +Firefox. Firefox builds its menu access keys from the localised menu names, so German reserves +`Alt+A`, `Alt+C` and `Alt+X`, which collide with archive, configuration and next session. Other +locales will differ again, and enumerating them all is not realistic. + +Two ways out, for a human to choose: + +1. **Keep `Alt` plus a letter.** Cheapest, and the collision only bites Firefox users in a + non-English interface, which is a small slice. The risk is a European user pressing `Alt+C` and + getting a browser menu instead of the configuration panel, with nothing to tell them why. +2. **Move to `Alt+Shift` plus a letter.** No browser menu claims a three-key chord, which is why + Linear uses that shape. It is correct in every locale. The cost is ergonomics: a three-key chord + is worse for a shortcut you press many times a session, and every label grows. + +The related open item is that the hook matches physical key positions (`event.code`) while the +sheet prints US letter legends. On AZERTY the key labelled `Z` reports `KeyW`, so the sheet says +`Alt+Z` while the user's key cap says `W`. Fixing that properly needs +`navigator.keyboard.getLayoutMap()` to derive the legend from the active layout. + +## Follow-up, not in this PR + +- The session tab menu still has no key column. Rename, Archive and Close each have a key. +- The session search box placeholder does not name `Alt+K`. +- The stop button does not name `Esc`. +- The composer hint row shows send and newline but not `Shift+Enter`. +- The connection dock has the same Approve-and-Deny gesture as the approval card and shows + neither key. +- The sheet is not searchable. Linear's is, and with forty-five rows ours will want it + eventually. + +## Verified + +Unit tests: `useSessionShortcuts.test.ts` 22 pass including the browser-key guard, +`ApprovalCard.test.tsx` 11 pass including both overlay guards, `shortcuts.test.ts` 13 pass on +the registry and its ARIA output. Suite totals: agenta-shared 440, agenta-chat 568, oss +AgentChatSlice 287. Every type-check and every package lint is clean. + +Both overlay regression tests were checked by removing the guard they cover and confirming +that test, and only that test, fails. + +Storybook builds clean and all eight stories render in light and dark with no page errors. + +On the live EE dev stack at port 8780, driven through Chrome: + +- The keyboard button is the last control in the playground top bar and carries + `aria-keyshortcuts="?"`. +- `?` opens the sheet from the page and is ignored while the caret is in the composer. +- Escape closes the sheet. +- The sheet renders 1040px wide in three columns with all twelve groups and no scrollbar. +- `Alt+C` collapses the configuration panel and restores it. The `»` button reports + `aria-keyshortcuts="Alt+C"` and the files caret reports `Alt+O`. +- The only console errors are PostHog 404s, which this dev stack has without the feature. + +## Not verified, and cannot be from here + +- The Alt chords on a real Windows browser and a real Linux desktop. On Linux a window manager + can claim an Alt chord before the browser sees it, and no page code can help. diff --git a/docs/docs/reference/agents/01-agent-configuration.mdx b/docs/docs/reference/agents/01-agent-configuration.mdx index e14f7acdd1..2d9c902187 100644 --- a/docs/docs/reference/agents/01-agent-configuration.mdx +++ b/docs/docs/reference/agents/01-agent-configuration.mdx @@ -59,7 +59,7 @@ Which providers, deployments, and connection modes each harness can reach is ser GET /api/workflows/catalog/harnesses/{harness} ``` -In the shipped table, `pi_core` and `pi_agenta` reach `openai`, `anthropic`, `gemini`, `mistral`, `groq`, `minimax`, `together_ai`, and `openrouter`, with deployments `direct` and `custom`. `claude` reaches `anthropic` only, with deployments `direct`, `custom`, `bedrock`, and `vertex_ai`, and selects its model by alias rather than by a `provider/model` string. +In the shipped table, `pi_core` reaches `openai`, `anthropic`, `gemini`, `mistral`, `groq`, `minimax`, `together_ai`, and `openrouter`, with deployments `direct` and `custom`. `claude` reaches `anthropic` only, with deployments `direct`, `custom`, `bedrock`, and `vertex_ai`, and selects its model by alias rather than by a `provider/model` string. ## tools @@ -195,18 +195,17 @@ Each entry is an inline skill package. See [Skills](/concepts/skills). | Field | Type | Required | Default | Description | |---|---|---|---|---| -| `kind` | `"pi_core"` \| `"pi_agenta"` \| `"claude"` \| `"codex"` | No | `"pi_core"` | Which coding agent to drive. | +| `kind` | `"pi_core"` \| `"claude"` \| `"codex"` | No | `"pi_core"` | Which coding agent to drive. | | `permissions` | object | No | see [harness.permissions](#harnesspermissions) | Tool-use gating posture, applied by harnesses that gate. | | `extras` | object | No | `{}` | Per-harness knobs passed through unchanged. For Pi, `system` replaces the base system prompt and `append_system` adds to it. Both are independent of `instructions.agents_md`. | | `kind` | Display name | Versioned slug | |---|---|---| | `pi_core` | Pi | `agenta:harness:pi_core:v0` | -| `pi_agenta` | Pi (Agenta) | `agenta:harness:pi_agenta:v0` | | `claude` | Claude Code | `agenta:harness:claude:v0` | | `codex` | Codex | `agenta:harness:codex:v0` | -`pi_agenta` runs the same engine as `pi_core` with Agenta's own skills, tools, and base instructions forced on. +`pi_agenta` was an experimental Pi variant, removed in August 2026. A stored config that still carries the value runs as `pi_core`. ### harness.permissions diff --git a/hosting/kubernetes/helm/Chart.yaml b/hosting/kubernetes/helm/Chart.yaml index a6610c6b54..ed15cff1df 100644 --- a/hosting/kubernetes/helm/Chart.yaml +++ b/hosting/kubernetes/helm/Chart.yaml @@ -2,8 +2,8 @@ apiVersion: v2 name: agenta description: A Helm chart for deploying Agenta (OSS or EE) on Kubernetes type: application -version: 0.114.3 -appVersion: "v0.114.3" +version: 0.114.4 +appVersion: "v0.114.4" keywords: - agenta - llm diff --git a/sdks/python/agenta/__init__.py b/sdks/python/agenta/__init__.py index e8f3cf9754..aab28899ea 100644 --- a/sdks/python/agenta/__init__.py +++ b/sdks/python/agenta/__init__.py @@ -56,7 +56,6 @@ # `agenta.Message` already names the prompt message type; import the agents one from # `agenta.sdk.agents` when needed. from .sdk.agents import ( # noqa: F401 - AgentaHarness, AgentTemplate, ClaudeHarness, Environment, diff --git a/sdks/python/agenta/sdk/agents/__init__.py b/sdks/python/agenta/sdk/agents/__init__.py index 915b9ad134..3efe9f6d50 100644 --- a/sdks/python/agenta/sdk/agents/__init__.py +++ b/sdks/python/agenta/sdk/agents/__init__.py @@ -21,7 +21,6 @@ """ from .adapters import ( - AgentaHarness, ClaudeHarness, CodexHarness, LocalBackend, @@ -55,7 +54,6 @@ UnsupportedProviderError, ) from .dtos import ( - AgentaAgentTemplate, AgentTemplate, AgentTemplateShapeError, Event, @@ -68,6 +66,7 @@ HarnessCapabilities, HarnessIdentity, HarnessKind, + InvalidHarnessKindError, InvalidPermissionDefaultError, Message, NetworkEgress, @@ -166,7 +165,6 @@ "PiAgentTemplate", "ClaudeAgentTemplate", "CodexAgentTemplate", - "AgentaAgentTemplate", "HarnessKind", "HarnessIdentity", "HARNESS_IDENTITIES", @@ -276,6 +274,7 @@ "LocalSandboxNotAllowedError", "UnsupportedHarnessError", "ToolResolutionError", + "InvalidHarnessKindError", "InvalidPermissionDefaultError", "AgentTemplateShapeError", # Adapters @@ -284,6 +283,5 @@ "PiHarness", "ClaudeHarness", "CodexHarness", - "AgentaHarness", "make_harness", ] diff --git a/sdks/python/agenta/sdk/agents/adapters/__init__.py b/sdks/python/agenta/sdk/agents/adapters/__init__.py index 590e8cded2..2a1e3dc6f1 100644 --- a/sdks/python/agenta/sdk/agents/adapters/__init__.py +++ b/sdks/python/agenta/sdk/agents/adapters/__init__.py @@ -2,7 +2,7 @@ - Backend adapters: ``SandboxAgentBackend`` (sandbox-agent over ACP), ``LocalBackend`` (standalone SDK runs; not yet implemented). -- Harness adapters: ``PiHarness``, ``ClaudeHarness``, ``CodexHarness``, ``AgentaHarness`` +- Harness adapters: ``PiHarness``, ``ClaudeHarness``, ``CodexHarness`` (+ ``make_harness``). - HTTP/browser protocol adapters live in subpackages, e.g. ``adapters.vercel``. @@ -10,7 +10,6 @@ """ from .harnesses import ( - AgentaHarness, ClaudeHarness, CodexHarness, PiHarness, @@ -25,6 +24,5 @@ "PiHarness", "ClaudeHarness", "CodexHarness", - "AgentaHarness", "make_harness", ] diff --git a/sdks/python/agenta/sdk/agents/adapters/agenta_builtins.py b/sdks/python/agenta/sdk/agents/adapters/agenta_builtins.py index dd346945e6..a576bff325 100644 --- a/sdks/python/agenta/sdk/agents/adapters/agenta_builtins.py +++ b/sdks/python/agenta/sdk/agents/adapters/agenta_builtins.py @@ -1,41 +1,31 @@ -"""The Agenta harness's forced defaults: the things ``AgentaHarness`` always applies. -(``ClaudeHarness`` shares the AGENTS.md preamble and forced platform skills; the persona -remains Pi-only — see :mod:`.harnesses`.) - -``AgentaHarness`` is Pi with an opinion. It is the same engine as :class:`PiHarness`, but -every run carries a fixed set of Agenta-shipped extras the author cannot turn off: - -- a base **persona** appended to Pi's system prompt (``AGENTA_FORCED_APPEND_SYSTEM``), -- a base **AGENTS.md preamble** the author's instructions are appended to (``AGENTA_PREAMBLE``), -- a set of **forced platform skills** (``AGENTA_FORCED_SKILLS``). - -The forced platform skills are the actually-forced part of "forced skills". The default agent -config template embeds the platform default skill by reserved ``__ag__*`` slug, but that embed -only rides the *default* template: a custom ``pi_agenta`` config that drops the embed would -otherwise lose the platform skill entirely. To make "forced" mean forced, ``AgentaHarness`` -unions ``AGENTA_FORCED_SKILLS`` into every run's skills via :func:`force_skills`, regardless of -what the author's config carries. The canonical skill content lives here (in the SDK, the lowest -layer); the server-side ``StaticWorkflowCatalog`` imports the same constant so the embed path -and the forced path stay one source of truth. - -Two layers, kept distinct on purpose (matching Pi's own split, see :class:`PiAgentTemplate`): -the *persona* is an ``append_system`` (changes Pi's base prompt), while *project conventions* -belong in ``AGENTS.md``. ``AGENTA_PREAMBLE`` is the AGENTS.md layer; ``AGENTA_FORCED_APPEND_SYSTEM`` -is the persona layer. - -One exception to "the Agenta harness's defaults": :func:`gateway_guidance` and -:func:`compose_gateway_guidance` are cross-harness. Every harness gets the same two derived -gateway tools, so every harness gets their instructions, and all four adapters import from -here. They live beside :func:`compose_instructions` because that function has to interleave -them with ``AGENTA_PREAMBLE``, which no other module owns. +"""Agenta-shipped agent content: the platform skills and the cross-harness gateway guidance. + +Two things live here: + +- The **platform skills** (getting started, build-an-agent) as concrete inline packages. The + canonical skill content is defined here (the SDK, the lowest layer); the server-side + ``StaticWorkflowCatalog`` imports the same constants so the embed path and the catalog stay + one source of truth. +- The **gateway guidance** (:func:`gateway_guidance` / :func:`compose_gateway_guidance`), + which is cross-harness: every harness gets the same two derived gateway tools, so every + harness gets their instructions, and all adapters import it from here. + +The ``pi_agenta`` harness (Pi plus a forced Agenta overlay: a preamble, a persona, forced +skills) was an experiment and was removed on 2026-08-29; the overlay constants and helpers +went with it. """ from __future__ import annotations -from typing import List, Optional, Sequence +from typing import Optional, Sequence from ..flags import ordered_operations_enabled +from typing import TYPE_CHECKING + from ..skills import SkillFile, SkillTemplate + +if TYPE_CHECKING: # circular at runtime: dtos imports skills, adapters import dtos + from ..dtos import GatewayGuidance from .agent_templates import build_agent_template_skill_files # Read once, at import, exactly like the op catalog builds its tool descriptions. The skill @@ -46,29 +36,6 @@ # skills list it meant to append to. _ORDERED = ordered_operations_enabled() -# The base AGENTS.md preamble. The author's own ``instructions`` are appended after this, so -# the final AGENTS.md is ``AGENTA_PREAMBLE`` + the author's project conventions. -# -# TODO(product): replace this placeholder with the real Agenta AGENTS.md preamble. -AGENTA_PREAMBLE = """\ -# Agenta agent - -You are an agent running on the Agenta platform. The instructions below are Agenta's -baseline; the user's own instructions follow and take precedence where they are more -specific. - -- Prefer the tools and skills provided to you over guessing. -- When a skill matches the task, read its SKILL.md fully before acting. -- Keep answers grounded in what the tools and skills actually return.""" - -# The base persona, always appended to Pi's built-in system prompt (never replaces it). This -# is the "who the agent is" layer, distinct from the AGENTS.md project-context layer above. -# -# TODO(product): replace this placeholder with the real Agenta persona framing. -AGENTA_FORCED_APPEND_SYSTEM = """\ -You are an Agenta agent. Be precise, cite what your tools and skills return, and do not -fabricate results.""" - # Reserved slug of the platform default skill. The default agent config template embeds the # skill by this slug; the server-side StaticWorkflowCatalog resolves the slug to the # SkillTemplate below. Kept here so the catalogue and the forced path share one slug constant. @@ -157,14 +124,14 @@ "tools": [], "mcps": [], "skills": [], - "harness": { "kind": "pi_agenta" }, + "harness": { "kind": "pi_core" }, "runner": { "kind": "sidecar", "permissions": { "default": "allow_reads" } }, "sandbox": { "kind": "local" } } ``` -The example above shows one common setup; your own `harness` may be `pi_agenta`, `claude`, or -`pi_core`. Whatever it is, keep `harness`, `runner`, `sandbox`, and `llm` as they are unless the +The example above shows one common setup; your own `harness` may be `pi_core`, `claude`, or +`codex`. Whatever it is, keep `harness`, `runner`, `sandbox`, and `llm` as they are unless the user explicitly asks to change one. ## The fields you decide @@ -172,8 +139,7 @@ ### instructions `instructions.agents_md` — a Markdown string, your AGENTS.md: who you are and what you do. Write -only your own project conventions here — the platform supplies its own baseline framing (on -`pi_agenta` and `claude`, a fixed Agenta preamble is prepended automatically). One or two +only your own project conventions here. One or two sentences for a simple agent; an explicit numbered procedure for a multi-tool or scheduled one (see the instruction-writing section of SKILL.md). @@ -183,7 +149,7 @@ to change the model, provider, or connection. The rules below matter only when they do ask: - `model` — the model. How you NAME it depends on the harness (this is the trap): - - `pi_core` / `pi_agenta`: a real model id, e.g. `gpt-5.5` or `anthropic/claude-...` + - `pi_core`: a real model id, e.g. `gpt-5.5` or `anthropic/claude-...` (provider/id selection). - `claude`: an alias — `default`, `sonnet`, `opus`, or `haiku` — never a raw model id. - `provider` — the provider family (`openai`, `anthropic`, ...); inferred from the model string @@ -299,7 +265,7 @@ ## The execution parts (keep as-is unless asked) -- `harness` — `{ "kind": "pi_core" | "pi_agenta" | "claude", "permissions": {...}, "extras": +- `harness` — `{ "kind": "pi_core" | "claude" | "codex", "permissions": {...}, "extras": {...} }`. `permissions` is `{ "default_mode": "default"|"acceptEdits"|"plan"| "bypassPermissions", "allow": [...], "ask": [...], "deny": [...] }`. The three rule lists name tools that run without asking, that ask first, and that are never allowed to run; each entry is @@ -405,7 +371,7 @@ - `harness.kind: "claude"` paired with a non-Anthropic `provider`. Claude reaches `anthropic` only. Bites at RUN time: the run's Model & Harness never resolves and the agent never runs. - A raw model id on the `claude` harness (Claude selects by alias) or an alias like `sonnet` on a - `pi_core`/`pi_agenta` harness (Pi selects by provider/id). Bites silently: the run falls back to + `pi_core` harness (Pi selects by provider/id). Bites silently: the run falls back to a default model with no error. Only `test_run`'s `resolved` block shows the fallback. - Naming an `@ag.embed` entry with a selector. An embed has no key, so no operation can address it. Leave those entries where they are. @@ -589,7 +555,7 @@ - `harness.kind: "claude"` paired with a non-Anthropic `provider`. Claude reaches `anthropic` only. Bites at RUN time: the run's Model & Harness never resolves and the agent never runs. - A raw model id on the `claude` harness (Claude selects by alias) or an alias like `sonnet` on a - `pi_core`/`pi_agenta` harness (Pi selects by provider/id). Bites silently: the run falls back + `pi_core` harness (Pi selects by provider/id). Bites silently: the run falls back to a default model with no error. Only `test_run`'s `resolved` block shows the fallback. - Sending a short `tools`/`skills`/`mcps` list. Bites on the NEXT run: lists replace wholesale, so every entry you left out is gone. @@ -1177,10 +1143,6 @@ ], ) -# Platform skills every pi_agenta run carries, regardless of the author's config. These are the -# actually-forced skills (see module docstring); unioned in by `force_skills`. -AGENTA_FORCED_SKILLS: List[SkillTemplate] = [GETTING_STARTED_WITH_AGENTA_SKILL] - def _join(*parts: Optional[str]) -> Optional[str]: """Join the non-empty parts with a blank line, or ``None`` when nothing remains.""" @@ -1205,8 +1167,9 @@ def gateway_guidance(integration_names: Sequence[str]) -> Optional[str]: return f"""\ ## Connected integrations -You can reach these integrations with two tools: `search_tools` and `run_tool`. -Configured integrations: {integrations}. +You can reach your integrations with two tools: `search_tools` and `run_tool`. +For instance, some of the integrations you have: {integrations}. Others may exist, and this +list can go stale — `search_tools` is the source of truth for what is connected right now. - Search once per task, with a concrete description of what you want to do. Never repeat an equivalent query — a second search that means the same thing returns the same results. @@ -1224,48 +1187,24 @@ def gateway_guidance(integration_names: Sequence[str]) -> Optional[str]: arguments — report it instead of looping.""" -def compose_gateway_guidance( - user: Optional[str], - integration_names: Sequence[str] = (), -) -> Optional[str]: - """One prompt layer with the gateway guidance placed before the author's own text. - - Every harness carries the guidance, not only the Agenta one: each gets the same two - derived tools, so a section added to one prompt surface would leave the others holding - two tools and no instructions for using them. Which layer carries it is the adapter's - choice, so ``user`` is whatever text that adapter puts the guidance in front of: the - instructions file for the file-based harnesses, and ``append_system`` for Pi, whose - AGENTS.md is purely authored. +def gateway_guidance_field( + integration_names: Sequence[str], + carrier: str, +) -> Optional["GatewayGuidance"]: + """The ``gatewayGuidance`` wire field, or ``None`` when the agent has no connection. + + Every harness carries the guidance, not only one: each gets the same two derived tools, + so guidance on one prompt surface alone would leave the others holding two tools and no + instructions for using them. ``carrier`` stays the adapter's choice (the instructions + file for the file-based harnesses, ``append_system`` for Pi, whose AGENTS.md is purely + authored) — but the SPLICING now happens in the runner, at environment build time, so the + integration names stay out of the session fingerprint and adding an integration no longer + evicts a warm session. The names read as examples, so a list that goes stale mid-session + stays honest until the next cold or reopened session refreshes it. """ - return _join(gateway_guidance(integration_names), user) - - -def compose_instructions( - user: Optional[str], - integration_names: Sequence[str] = (), -) -> Optional[str]: - """The AGENTS.md the Agenta harness ships: the base preamble, then the gateway guidance - when the agent has a connection, then the author's instructions.""" - return _join(AGENTA_PREAMBLE, compose_gateway_guidance(user, integration_names)) - - -def compose_append_system(user: Optional[str]) -> Optional[str]: - """The ``append_system`` the harness ships: the forced base persona with the author's own - ``append_system`` appended after it.""" - return _join(AGENTA_FORCED_APPEND_SYSTEM, user) - - -def force_skills(skills: List[SkillTemplate]) -> List[SkillTemplate]: - """Union the author's skills with the forced platform skills, de-duplicated by name. - - The author's skills come first and win on a name clash (a config that already carries the - resolved platform skill — e.g. via the default template's embed — is not doubled), then any - forced platform skill not already present is appended. This is what makes the ``_agenta`` - platform skill actually forced on a custom ``pi_agenta`` config that drops the embed.""" - seen = {skill.name for skill in skills} - out: List[SkillTemplate] = list(skills) - for forced in AGENTA_FORCED_SKILLS: - if forced.name not in seen: - seen.add(forced.name) - out.append(forced) - return out + text = gateway_guidance(integration_names) + if not text: + return None + from ..dtos import GatewayGuidance + + return GatewayGuidance(text=text, carrier=carrier) diff --git a/sdks/python/agenta/sdk/agents/adapters/harnesses.py b/sdks/python/agenta/sdk/agents/adapters/harnesses.py index e1764d1c11..b83a1ec832 100644 --- a/sdks/python/agenta/sdk/agents/adapters/harnesses.py +++ b/sdks/python/agenta/sdk/agents/adapters/harnesses.py @@ -9,11 +9,9 @@ shared permission plan. - **claude** has no built-in tools (they are a Pi concept), delivers tools over MCP, and receives the same runner permission plan. -- **pi_agenta** is Pi with an opinion: the same engine and config shape, plus a base AGENTS.md - preamble and a persona (see :mod:`.agenta_builtins`). - Skills ride the neutral config as resolved inline packages. Pi and Agenta install them - through Pi skill dirs; Claude carries them so the runner can write project-local - `.claude/skills` packages. Seeding platform default skills is a separate workstream. +- Skills ride the neutral config as resolved inline packages. Pi installs them through Pi + skill dirs; Claude carries them so the runner can write project-local `.claude/skills` + packages. Seeding platform default skills is a separate workstream. The backend below stays pure plumbing; this layer owns the harness knowledge. """ @@ -23,7 +21,6 @@ from typing import Any, Dict, List, Type from ..dtos import ( - AgentaAgentTemplate, ClaudeAgentTemplate, CodexAgentTemplate, HarnessKind, @@ -32,12 +29,7 @@ ) from ..interfaces import Environment, Harness from ..tools.models import ToolSpec, coerce_tool_spec -from .agenta_builtins import ( - compose_append_system, - compose_gateway_guidance, - compose_instructions, - force_skills, -) +from .agenta_builtins import gateway_guidance_field def _opt_str(value: Any) -> Any: @@ -82,9 +74,9 @@ def _to_harness_config(self, config: SessionConfig) -> PiAgentTemplate: permission_default=config.permission_default, harness_permissions=config.agent.harness_permissions, system=_opt_str(extras.get("system")), - append_system=compose_gateway_guidance( - _opt_str(extras.get("append_system")), - config.gateway_integration_names, + append_system=_opt_str(extras.get("append_system")), + gateway_guidance=gateway_guidance_field( + config.gateway_integration_names, "appendSystemPrompt" ), ) @@ -102,8 +94,9 @@ def _to_harness_config(self, config: SessionConfig) -> ClaudeAgentTemplate: # adapter) renders `.claude/settings.json` as a generic `harnessFiles` entry. No # claude-specific parsing happens here; the runner just writes the files into the cwd. return ClaudeAgentTemplate( - agents_md=compose_gateway_guidance( - config.agent.instructions, config.gateway_integration_names + agents_md=config.agent.instructions, + gateway_guidance=gateway_guidance_field( + config.gateway_integration_names, "agentsMd" ), model=config.agent.model, resolved_connection=config.resolved_connection, @@ -130,8 +123,9 @@ def _to_harness_config(self, config: SessionConfig) -> CodexAgentTemplate: # adapter) renders `.codex/config.toml` as a generic `harnessFiles` entry. No # codex-specific parsing happens here; the runner just writes the files into the cwd. return CodexAgentTemplate( - agents_md=compose_gateway_guidance( - config.agent.instructions, config.gateway_integration_names + agents_md=config.agent.instructions, + gateway_guidance=gateway_guidance_field( + config.gateway_integration_names, "agentsMd" ), model=config.agent.model, resolved_connection=config.resolved_connection, @@ -145,49 +139,10 @@ def _to_harness_config(self, config: SessionConfig) -> CodexAgentTemplate: ) -class AgentaHarness(Harness): - """Pi with an Agenta opinion. Same engine as :class:`PiHarness`, but every run carries the - forced Agenta extras (see :mod:`.agenta_builtins`): a base AGENTS.md preamble the author's - instructions are appended to, and a forced persona ``append_system``. The - author's own Pi ``harness.extras`` (``system`` / ``append_system``) still apply, layered - after the forced bits. The author's resolved inline skills ride the neutral config, and the - forced platform skill(s) are unioned in (de-duped by name) so a custom config that drops the - default template's ``_agenta`` embed still carries the platform skill.""" - - harness_type = HarnessKind.AGENTA - - def _to_harness_config(self, config: SessionConfig) -> AgentaAgentTemplate: - # The author's Pi options still apply; the pi_agenta harness reads the same harness - # `extras` as PiHarness (it drives Pi) and layers its forced extras on top. - extras = config.agent.harness_extras - return AgentaAgentTemplate( - agents_md=compose_instructions( - config.agent.instructions, config.gateway_integration_names - ), - model=config.agent.model, - # See PiHarness: thread the structured ref so a named custom connection's {mode, slug} - # reaches the /run wire and the runner can build its models.json plan. - model_ref=config.agent.model_ref, - resolved_connection=config.resolved_connection, - tool_specs=list(config.tool_specs), - tool_callback=config.tool_callback, - mcp_servers=list(config.mcp_servers), - # Force the platform skill(s) into every run, de-duped by name. A custom config that - # drops the default template's `_agenta` embed still gets the platform skill. - skills=force_skills(list(config.agent.skills)), - sandbox_permission=config.agent.sandbox_permission, - permission_default=config.permission_default, - harness_permissions=config.agent.harness_permissions, - system=_opt_str(extras.get("system")), - append_system=compose_append_system(_opt_str(extras.get("append_system"))), - ) - - _HARNESSES: Dict[HarnessKind, Type[Harness]] = { HarnessKind.PI: PiHarness, HarnessKind.CLAUDE: ClaudeHarness, HarnessKind.CODEX: CodexHarness, - HarnessKind.AGENTA: AgentaHarness, } diff --git a/sdks/python/agenta/sdk/agents/adapters/sandbox_agent.py b/sdks/python/agenta/sdk/agents/adapters/sandbox_agent.py index 5ffa4a55f9..a7f654bda5 100644 --- a/sdks/python/agenta/sdk/agents/adapters/sandbox_agent.py +++ b/sdks/python/agenta/sdk/agents/adapters/sandbox_agent.py @@ -125,14 +125,13 @@ def stream(self, messages: Sequence[Message]) -> AgentStream: class SandboxAgentBackend(Backend): - """The sandbox-agent engine: a harness over ACP through the TS runner. Pi, Claude, Codex, and Agenta.""" + """The sandbox-agent engine: a harness over ACP through the TS runner. Pi, Claude, and Codex.""" supported_harnesses = frozenset( { HarnessKind.PI, HarnessKind.CLAUDE, HarnessKind.CODEX, - HarnessKind.AGENTA, } ) diff --git a/sdks/python/agenta/sdk/agents/capabilities.py b/sdks/python/agenta/sdk/agents/capabilities.py index 2e45dd0efa..495e67b485 100644 --- a/sdks/python/agenta/sdk/agents/capabilities.py +++ b/sdks/python/agenta/sdk/agents/capabilities.py @@ -29,8 +29,6 @@ Bedrock/Vertex. The runner passes the selected model id through to Claude Code and lets the configured backend fail loudly if it rejects it. - **Codex** reaches openai only, direct, through managed keys or subscription OAuth. -- **pi_agenta** is Pi under the hood (Pi with Agenta's forced opinion), so it shares - ``pi_core``'s reach. The sibling ``docs/design/agent-workflows/projects/harness-capabilities/`` project owns the general capability-table mechanism; this module is the provider/model/auth contribution @@ -392,15 +390,6 @@ def _derive_default_models(self) -> "HarnessConnectionCapabilities": models=_pi_models(), model_catalog=_model_catalog("pi_core"), ), - "pi_agenta": HarnessConnectionCapabilities( - # See ``pi_core``: ``custom`` is UI-surface only; ``harness_allows_pair`` is authoritative. - providers=list(PI_VAULT_PROVIDERS) + list(PI_SUBSCRIPTION_PROVIDERS), - deployments=["direct", "custom"], - connection_modes=list(_ALL_MODES), - model_selection="provider/id", - models=_pi_models(), - model_catalog=_model_catalog("pi_agenta"), - ), "claude": HarnessConnectionCapabilities( providers=["anthropic"], deployments=["direct", "custom", "bedrock", "vertex_ai", "vertex"], @@ -491,7 +480,7 @@ def harness_allows_deployment(harness: str, deployment: str) -> bool: """Whether ``harness`` can CONSUME the resolved ``deployment`` in v1. A harness with no entry is unknown, so it gets no capability (closed). The cloud surfaces - are allowed only when the harness lists them as consumable. ``pi_core``/``pi_agenta`` list + are allowed only when the harness lists them as consumable. ``pi_core`` lists ``direct`` and ``custom`` (the OpenAI-compatible surface); Claude also lists ``bedrock``/``vertex_ai``. """ @@ -509,7 +498,6 @@ def harness_allows_deployment(harness: str, deployment: str) -> bool: # absent here accepts no ``custom`` deployment. HARNESS_CUSTOM_DEPLOYMENT_PROVIDERS: Dict[str, str] = { "pi_core": "openai", - "pi_agenta": "openai", "claude": "anthropic", } @@ -527,8 +515,8 @@ def harness_allows_pair(harness: str, provider: str, deployment: str) -> bool: The allowed triples: - - ``pi_core``/``pi_agenta`` + ``openai`` + ``direct`` or ``custom`` -> allowed; - - ``pi_core``/``pi_agenta`` + any other family + ``custom`` -> rejected; + - ``pi_core`` + ``openai`` + ``direct`` or ``custom`` -> allowed; + - ``pi_core`` + any other family + ``custom`` -> rejected; - ``claude`` + ``anthropic`` + ``direct``/``custom``/``bedrock``/``vertex_ai`` -> allowed; - ``claude`` + ``openai`` + anything -> rejected (Claude reaches anthropic only); - unknown harness -> rejected. diff --git a/sdks/python/agenta/sdk/agents/dtos.py b/sdks/python/agenta/sdk/agents/dtos.py index dd74fcefab..950f83b4fb 100644 --- a/sdks/python/agenta/sdk/agents/dtos.py +++ b/sdks/python/agenta/sdk/agents/dtos.py @@ -46,21 +46,61 @@ class HarnessKind(str, Enum): """The coding agent program a run drives. A backend declares which it supports. - ``pi_core`` is plain Pi; ``pi_agenta`` is Pi with Agenta's forced skills, prompt, and - policy. Both drive the same ``pi`` ACP agent in the runner; ``claude`` drives Claude Code. + ``pi_core`` is Pi; ``claude`` drives Claude Code; ``codex`` drives Codex. """ PI = "pi_core" CLAUDE = "claude" - AGENTA = "pi_agenta" CODEX = "codex" @classmethod def coerce(cls, value: "HarnessKind | str") -> "HarnessKind": - """Accept either an enum or a loose string (the playground sends a string).""" + """Accept either an enum or a loose string (the playground sends a string). + + Raises :class:`InvalidHarnessKindError` for anything else, so a value the runtime + cannot read fails as a named 400 rather than the enum's bare ``ValueError``. + """ if isinstance(value, cls): return value - return cls(str(value).lower()) + normalized = str(value).lower() + # ``pi_agenta`` was a short-lived experiment (Pi plus a forced Agenta overlay), removed + # 2026-08-29. Revisions saved while it existed still carry the value, so reading maps it + # to plain Pi instead of refusing to load the config. + if normalized == "pi_agenta": + normalized = "pi_core" + try: + return cls(normalized) + except ValueError as exc: + raise InvalidHarnessKindError(value) from exc + + +class InvalidHarnessKindError(ErrorStatus, ValueError): + """``harness.kind`` names a harness this runtime does not have. + + Maps to HTTP 400 so the caller is told the field, the value it sent, and the values that + exist. Before this, a malformed kind reached ``make_harness`` as the enum's bare + ``ValueError`` and surfaced as an unhandled 500 whose body was the Python repr, and the + commit that stored it answered 200 (finding F4). + + It is also a ``ValueError``, because the enum has always raised one here and callers + (and tests) guard ``coerce`` on that type. Inheriting both keeps those guards working + while the middleware renders the coded status. + """ + + code: int = 400 + type: str = f"{ERRORS_BASE_URL}#v0:agent:invalid-harness-kind" + + def __init__(self, value: Any) -> None: + allowed = ", ".join(sorted(kind.value for kind in HarnessKind)) + super().__init__( + code=self.code, + type=self.type, + message=( + f"invalid harness.kind ({type(value).__name__}) {value!r}; " + f"expected one of {allowed}" + ), + ) + self.value = value # --------------------------------------------------------------------------- @@ -72,7 +112,7 @@ def coerce(cls, value: "HarnessKind | str") -> "HarnessKind": # ``engines/running/interfaces.py``). The namespace is ``harness`` and the trailing ``v0`` is # bumped only when the harness contract shape breaks. This is purely the INTERFACE identity the # agent_template schema advertises; the stored/wire harness VALUE stays the bare enum string -# (``pi_core`` / ``pi_agenta`` / ``claude``), which the runner reads as the runtime selector. +# (``pi_core`` / ``claude`` / ``codex``), which the runner reads as the runtime selector. class HarnessIdentity(BaseModel): @@ -97,11 +137,6 @@ class HarnessIdentity(BaseModel): slug=f"agenta:harness:{HarnessKind.PI.value}:v0", name="Pi", ), - HarnessIdentity( - value=HarnessKind.AGENTA.value, - slug=f"agenta:harness:{HarnessKind.AGENTA.value}:v0", - name="Pi (Agenta)", - ), HarnessIdentity( value=HarnessKind.CLAUDE.value, slug=f"agenta:harness:{HarnessKind.CLAUDE.value}:v0", @@ -698,6 +733,26 @@ def from_params( # --------------------------------------------------------------------------- +class GatewayGuidance(BaseModel): + """The derived gateway-tools instruction section, carried as its own wire field. + + It used to be composed INTO the prompt strings (``append_system`` for Pi, ``agents_md`` + for the file-based harnesses). That put the integration NAMES inside the session + fingerprint, so adding a second integration evicted a warm session for a one-word prompt + change. As a separate field the runner splices it into ``carrier`` when it BUILDS an + environment, and deliberately excludes it from the fingerprint: the text refreshes + whenever a session is built or reopened, and never evicts one on its own. The wording + presents the names as examples ("for instance"), so a list that goes stale mid-session + stays honest. + """ + + text: str + carrier: Literal["appendSystemPrompt", "agentsMd"] + + def to_wire(self) -> Dict[str, Any]: + return {"text": self.text, "carrier": self.carrier} + + class HarnessAgentTemplate(BaseModel): """Base for a harness-specific config. A Harness produces one of these from the neutral config; a backend plumbs it as-is, with no business logic about how the harness works. @@ -731,6 +786,9 @@ class HarnessAgentTemplate(BaseModel): # it into files for the wire (see :meth:`wire_harness_files`); the raw slice does not ride the # wire. harness_permissions: Dict[str, Any] = Field(default_factory=dict) + # The derived gateway-tools guidance, set by the adapter when the agent has at least one + # gateway connection. Rides the wire as ``gatewayGuidance``; see :class:`GatewayGuidance`. + gateway_guidance: Optional[GatewayGuidance] = None @model_validator(mode="before") @classmethod @@ -774,6 +832,14 @@ def wire_prompt(self) -> Dict[str, Any]: by default; a harness that exposes prompt overrides (Pi) emits them here.""" return {} + def wire_gateway_guidance(self) -> Dict[str, Any]: + """The ``gatewayGuidance`` field for the ``/run`` payload. Omitted when the agent has + no gateway connection so a connection-free payload is unchanged (the golden wire + contract).""" + if not self.gateway_guidance: + return {} + return {"gatewayGuidance": self.gateway_guidance.to_wire()} + def wire_mcp(self) -> Dict[str, Any]: """The ``mcpServers`` field for the ``/run`` payload. Omitted when none are declared so a tool-free run's payload is unchanged (the golden wire contract).""" @@ -1074,14 +1140,6 @@ def wire_harness_files(self) -> Dict[str, Any]: return {"harnessFiles": files} -class AgentaAgentTemplate(PiAgentTemplate): - """The Agenta harness's config. It *is* a Pi config (same engine, same tool delivery and - system-prompt layers). ``skills`` ride the inherited :meth:`wire_skills` seam as resolved - inline packages, not through ``wire_tools`` (skills are not tools).""" - - harness: ClassVar[HarnessKind] = HarnessKind.AGENTA - - # --------------------------------------------------------------------------- # The session bundle # --------------------------------------------------------------------------- @@ -1349,7 +1407,26 @@ def _parse_run_selection( ``harness`` from ``harness.kind``, ``sandbox`` from ``sandbox.kind``, and the runner policy from ``runner.permissions.default``. The kinds and permission mode are lower-cased so playground-supplied values match the bare runtime selectors.""" - harness = str(_section(params, "harness").get("kind") or defaults.harness).lower() + # An absent, null, or blank kind means "use the default", which is what every config that + # never set a harness relies on. Anything else must name a harness this runtime has, and a + # kind it cannot read is refused HERE, where the template is parsed, rather than travelling + # to `make_harness` and surfacing as an unhandled 500 (finding F4). The returned value keeps + # its stored spelling, so a legacy value still normalizes where it always did. + raw_harness = _section(params, "harness").get("kind") + if raw_harness is None or not str(raw_harness).strip(): + harness = str(defaults.harness).lower() + else: + resolved = HarnessKind.coerce(raw_harness) + # A STRING keeps its stored spelling, so a legacy value still normalizes exactly where + # it always did (`make_harness` maps `pi_agenta` to Pi; collapsing it here would change + # the harness identity a stored revision carries). A MEMBER has no spelling to keep, so + # it becomes its wire value: `str()` on one gives "HarnessKind.CLAUDE", which lower-cased + # to "harnesskind.claude" and made the SDK's own enum unusable as input. + harness = ( + str(raw_harness).lower() + if not isinstance(raw_harness, HarnessKind) + else resolved.value + ) sandbox = str(_section(params, "sandbox").get("kind") or defaults.sandbox).lower() permissions = _section(params, "runner").get("permissions") raw_default = permissions.get("default") if isinstance(permissions, dict) else None diff --git a/sdks/python/agenta/sdk/agents/model_catalog.py b/sdks/python/agenta/sdk/agents/model_catalog.py index c0214718d3..a42549a98b 100644 --- a/sdks/python/agenta/sdk/agents/model_catalog.py +++ b/sdks/python/agenta/sdk/agents/model_catalog.py @@ -202,7 +202,7 @@ def model_input_modalities( ) -> Optional[List[str]]: """Look up input modalities using the model id form accepted by ``harness``.""" entry: Optional[ModelCatalogEntry] - if harness in ("pi_core", "pi_agenta"): + if harness == "pi_core": catalog = pi_model_catalog() catalog_id = _catalog_id(provider, model_id) elif harness == "claude": @@ -235,7 +235,7 @@ def model_catalog_entries(harness: str) -> List[Dict[str, object]]: uses its curated model catalog. An unknown harness has an empty catalog (like the ``models`` map default). """ - if harness in ("pi_core", "pi_agenta"): + if harness == "pi_core": catalog = pi_model_catalog() elif harness == "claude": catalog = claude_model_catalog() diff --git a/sdks/python/agenta/sdk/agents/platform/gateway.py b/sdks/python/agenta/sdk/agents/platform/gateway.py index 05a53ef203..7fad25dbf5 100644 --- a/sdks/python/agenta/sdk/agents/platform/gateway.py +++ b/sdks/python/agenta/sdk/agents/platform/gateway.py @@ -104,10 +104,12 @@ def _derived_tool_specs(integration_names: Sequence[str]) -> List[CallbackToolSp covers every configured integration, because the model selects the integration in the arguments rather than through the tool name. - ``search_tools`` names the connected integrations in its own description. "Never invent - an integration name" is only actionable next to the list of real ones, and a tool - description is read at every call while the prompt guidance can fall out of a long - context. Names only, the same ones the guidance already carries. + The tool descriptions deliberately do NOT name the connected integrations. The names + live in the ``gatewayGuidance`` prompt section instead: ``customTools`` is part of the + session fingerprint, so a names sentence here made ADDING AN INTEGRATION evict the warm + session for a one-word description change. With the descriptions stable, the derived + pair is byte-identical for any integration set, and only the FIRST connection (which + genuinely adds the two tools) changes session config. Both carry ``permission: "allow"``. That is not an authorization decision. It only opens the coarse harness gate so the call reaches the runner; the real boundary is the runner's @@ -116,13 +118,6 @@ def _derived_tool_specs(integration_names: Sequence[str]) -> List[CallbackToolSp harness would resolve ``run_tool`` through the agent-wide mode and raise a second, meaningless approval card named ``run_tool`` before the runner saw the tool key at all. """ - # The resolver only runs with at least one entry, so the empty join is unreachable - # today. Kept total anyway: a dangling "Connected integrations: ." is model-facing text. - connected = ( - f" Connected integrations: {', '.join(sorted(integration_names))}." - if integration_names - else "" - ) return [ CallbackToolSpec( name="search_tools", @@ -133,7 +128,6 @@ def _derived_tool_specs(integration_names: Sequence[str]) -> List[CallbackToolSp "matches first — a cap, not the whole catalog. If the search fails, retry it " "once and no more. If nothing matched, search again with a more specific " "description of the task, then stop. Never invent an integration name." - f"{connected}" ), input_schema={ "type": "object", diff --git a/sdks/python/agenta/sdk/agents/platform/workflow.py b/sdks/python/agenta/sdk/agents/platform/workflow.py index 6b6e8b6172..8e5a6e7d2e 100644 --- a/sdks/python/agenta/sdk/agents/platform/workflow.py +++ b/sdks/python/agenta/sdk/agents/platform/workflow.py @@ -27,6 +27,7 @@ GatewayToolResolutionError, ReferenceToolConfig, ToolCallback, + disambiguate_tool_names, ) from agenta.sdk.utils.logging import get_module_logger @@ -62,6 +63,18 @@ async def resolve( # endpoint and its auth cannot diverge. authorization = self._connection.authorization() + # Resolve every model-visible name up front. Sanitizing can merge two distinct children + # onto one name ("Support Router" and "Support/Router" both become `Support_Router`), and + # a duplicate name silently shadows the earlier tool instead of erroring — so the second + # subagent would simply never be callable. This is the only place that sees siblings. + names_by_call_ref = disambiguate_tool_names( + [ + (tool_config.call_ref, tool_config.tool_name) + for tool_config in tools + if not _is_request_connection_workflow(tool_config) + ] + ) + seen: set[str] = set() tool_specs: list[CallbackToolSpec | ClientToolSpec] = [] for tool_config in tools: @@ -86,10 +99,16 @@ async def resolve( ) ) continue + resolved_name = names_by_call_ref[call_ref] tool_specs.append( CallbackToolSpec( - name=tool_config.tool_name, - description=tool_config.description or tool_config.tool_name, + name=resolved_name, + # The DESCRIPTION keeps the authored display name when there is one: that is + # what the model reads to decide whether to call this subagent, and the + # sanitized wire name may have lost the spacing that made it readable. + description=tool_config.description + or tool_config.name + or resolved_name, # Expand Agenta catalog pointers (``x-ag-type-ref``, e.g. ``messages``) into # concrete JSON Schema so the harness sees a real shape (an array WITH items, # not a bare ``x-ag-type-ref``) and can construct the call. Reference tools are diff --git a/sdks/python/agenta/sdk/agents/tools/__init__.py b/sdks/python/agenta/sdk/agents/tools/__init__.py index b98cc9864e..c1e55884f2 100644 --- a/sdks/python/agenta/sdk/agents/tools/__init__.py +++ b/sdks/python/agenta/sdk/agents/tools/__init__.py @@ -38,6 +38,7 @@ CodeToolConfig, CodeToolSpec, CompiledTool, + disambiguate_tool_names, GatewayConnectionPolicy, GatewayConnectionRef, GatewayConnectionResolution, @@ -54,6 +55,7 @@ ResolvedGatewayIntegration, ResolvedGatewayPolicy, ResolvedToolSet, + sanitize_tool_name, ToolCall, ToolCallback, ToolConfig, @@ -76,6 +78,8 @@ "CodeToolConfig", "ClientToolConfig", "ReferenceToolConfig", + "disambiguate_tool_names", + "sanitize_tool_name", "PlatformToolConfig", "ToolSpec", "CallbackToolSpec", diff --git a/sdks/python/agenta/sdk/agents/tools/models.py b/sdks/python/agenta/sdk/agents/tools/models.py index 034dac8509..85b5fae50a 100644 --- a/sdks/python/agenta/sdk/agents/tools/models.py +++ b/sdks/python/agenta/sdk/agents/tools/models.py @@ -2,7 +2,9 @@ from __future__ import annotations +import re from enum import Enum +from hashlib import sha1 from typing import Annotated, Any, Dict, List, Literal, Optional, Union from pydantic import ( @@ -20,6 +22,90 @@ def _empty_object_schema() -> Dict[str, Any]: return {"type": "object", "properties": {}} +#: The character class every major provider accepts for a tool name (`^[a-zA-Z0-9_.-]+$`). +#: OpenAI rejects the whole request when any tool violates it — not the one tool, the whole +#: `tools` array — so a single bad name breaks every run of the agent that owns it. +_TOOL_NAME_ALLOWED = re.compile(r"[^a-zA-Z0-9_.-]+") + + +def sanitize_tool_name(raw: Optional[str], *, fallback: str) -> str: + """Coerce an authored name into the provider's tool-name pattern. + + A subagent's model-visible name is a DISPLAY name the user typed, so it can carry spaces, + slashes, or anything else a person writes. Sending it unchanged made the provider refuse the + entire tool list with `Invalid 'tools[N].name'`, which bricks every run of the parent agent + until the child is renamed. Names like "Support Router" are an ordinary thing to type. + + The mapping is deterministic and stable, because the model sees this name and a name that + changed between turns would strand a conversation mid-tool-call: every disallowed run of + characters becomes one `_`, leading and trailing separators are trimmed, and an input that + survives none of that falls back to `fallback` (itself sanitized). Only the WIRE name is + touched; the display name is never rewritten. + """ + collapsed = _TOOL_NAME_ALLOWED.sub("_", (raw or "").strip()) + # Trim separators the collapse may have produced at either end. `.` and `-` are legal + # characters but a leading or trailing one reads as debris rather than a name. + trimmed = collapsed.strip("_.-") + if trimmed: + return trimmed + if raw is not None or fallback: + cleaned_fallback = _TOOL_NAME_ALLOWED.sub("_", fallback.strip()).strip("_.-") + if cleaned_fallback: + return cleaned_fallback + return "tool" + + +def disambiguate_tool_names(pairs: List[tuple]) -> Dict[str, str]: + """Map each `(identity, sanitized_name)` to a name unique across the list. + + Sanitizing can merge two distinct children onto one name — "Support Router" and + "Support/Router" both become `Support_Router` — and a duplicate tool name silently shadows + the earlier tool rather than erroring, so the second subagent would simply never be callable. + + Only colliding names are decorated, so the common case keeps the name the user recognizes. + The discriminator is a digest of the tool's own stable identity (its `call_ref`), NOT its + position: an ordinal would renumber when the author reorders or removes a tool, changing a + name the model may already have used earlier in the conversation. + + The digest starts short for readability and LENGTHENS until every name in the colliding group + is distinct. Six hex characters is only 24 bits, so two `call_ref` values can share a prefix; + if their base names also matched, the function would hand back one name for both entries and + the final uniqueness check would reject a configuration this helper promises is unique. Growing + the digest keeps the guarantee without reintroducing an ordinal: the length depends on the set + of colliding identities, never on their order, so the same configuration always yields the same + names. Two distinct identities cannot share the FULL digest, so the loop always terminates. + """ + counts: Dict[str, int] = {} + for _identity, name in pairs: + counts[name] = counts.get(name, 0) + 1 + + resolved: Dict[str, str] = {} + for name, count in counts.items(): + group = [identity for identity, item_name in pairs if item_name == name] + if count == 1: + resolved[group[0]] = name + continue + digests = {identity: _identity_digest(identity) for identity in group} + # Sorting makes the width a property of the SET, not of iteration order. + width = _shortest_distinct_prefix(sorted(digests.values())) + for identity in group: + resolved[identity] = f"{name}_{digests[identity][:width]}" + return resolved + + +def _identity_digest(identity: str) -> str: + return sha1(identity.encode("utf-8")).hexdigest() + + +def _shortest_distinct_prefix(digests: List[str], start: int = 6) -> int: + """The smallest prefix length (from `start`) at which every digest differs.""" + longest = max((len(digest) for digest in digests), default=start) + for width in range(start, longest + 1): + if len({digest[:width] for digest in digests}) == len(digests): + return width + return longest + + # Layer-3 per-tool permission: ``allow`` runs with no prompt, ``ask`` raises a # human-in-the-loop request, ``deny`` never runs. Absent means "inherit the runner policy". Permission = Literal["allow", "ask", "deny"] @@ -317,8 +403,15 @@ def _check_axis(self) -> "ReferenceToolConfig": @property def tool_name(self) -> str: - """The model-visible name; defaults to the workflow slug when none is authored.""" - return self.name or self.slug + """The model-visible name; defaults to the workflow slug when none is authored. + + Sanitized to the provider's tool-name pattern, because the authored `name` is a DISPLAY + name a person typed and may contain spaces or punctuation the provider refuses. The + display name itself is never rewritten — only this wire value. Collisions between two + children that sanitize alike are resolved by the caller building the tool list, which is + the only place that can see siblings. + """ + return sanitize_tool_name(self.name, fallback=self.slug) @property def call_ref(self) -> str: diff --git a/sdks/python/agenta/sdk/agents/tools/resolver.py b/sdks/python/agenta/sdk/agents/tools/resolver.py index d9d9f057e1..b8b605960b 100644 --- a/sdks/python/agenta/sdk/agents/tools/resolver.py +++ b/sdks/python/agenta/sdk/agents/tools/resolver.py @@ -92,11 +92,15 @@ def _build_client_tool_spec(*, tool_config: ClientToolConfig) -> ClientToolSpec: ) -def _check_tool_name(name: str, seen: set[str]) -> None: +def _reject_reserved_tool_name(name: str) -> None: # The harness registers custom tools by name beside its built-ins, so a same-named custom # tool would silently replace the built-in the platform activates on every run. if name.strip().lower() in PI_BUILTIN_TOOL_NAMES: raise ReservedToolNameError(name) + + +def _check_tool_name(name: str, seen: set[str]) -> None: + _reject_reserved_tool_name(name) if name in seen: raise DuplicateToolNameError(name) seen.add(name) @@ -130,8 +134,19 @@ def _validate_declared_config_names(tool_configs: Sequence[ToolConfig]) -> None: seen: set[str] = set() for tool_config in tool_configs: name = _declared_config_name(tool_config) - if name is not None: - _check_tool_name(name, seen) + if name is None: + continue + if isinstance(tool_config, ReferenceToolConfig): + # A reference tool's model-visible name is DERIVED: an authored display name, + # sanitized to the provider's tool-name pattern. Two distinct children can therefore + # arrive here sharing one name ("Support Router" and "Support/Router" both sanitize + # to `Support_Router`) without either being a mistake. The workflow adapter gives + # them distinct names before they reach the wire, and `_validate_unique_names` still + # checks the result, so rejecting them here would refuse a valid configuration. The + # reserved-name check still applies — that one is about shadowing a built-in. + _reject_reserved_tool_name(name) + continue + _check_tool_name(name, seen) def _validate_unique_names(tool_specs: Sequence[ToolSpec]) -> None: diff --git a/sdks/python/agenta/sdk/agents/utils/wire.py b/sdks/python/agenta/sdk/agents/utils/wire.py index eb17b5a9db..77e495171b 100644 --- a/sdks/python/agenta/sdk/agents/utils/wire.py +++ b/sdks/python/agenta/sdk/agents/utils/wire.py @@ -155,6 +155,7 @@ def request_to_wire( "telemetry": trace.telemetry_to_wire() if trace else None, **config.wire_tools(), **config.wire_prompt(), + **config.wire_gateway_guidance(), **config.wire_mcp(), **config.wire_skills(), **config.wire_sandbox_permission(), diff --git a/sdks/python/agenta/sdk/agents/wire_models.py b/sdks/python/agenta/sdk/agents/wire_models.py index 0ab82af68f..a6acc80987 100644 --- a/sdks/python/agenta/sdk/agents/wire_models.py +++ b/sdks/python/agenta/sdk/agents/wire_models.py @@ -344,6 +344,13 @@ class WireGatewayIntegration(_WireModel): tools: Dict[str, WireGatewayTool] = Field(default_factory=dict) +class WireGatewayGuidance(_WireModel): + """The derived gateway-tools instruction section (``gatewayGuidance`` on the request).""" + + text: str + carrier: Literal["appendSystemPrompt", "agentsMd"] + + class WireGatewayPolicy(_WireModel): """The private compiled gateway policy (``gatewayPolicy`` on the request). @@ -501,7 +508,7 @@ class WireRunRequest(_WireModel): Every field is optional on the wire (the contract is implicitly all-optional), so the schema expresses "optional" while the producer's omit-when-empty behavior stays in ``wire.py`` and - is pinned by the golden fixtures. The harness selects the agent (``pi_core`` / ``pi_agenta`` + is pinned by the golden fixtures. The harness selects the agent (``pi_core`` / ``claude``); there is no engine selector on the wire (A3 removed the legacy backend). """ @@ -555,6 +562,12 @@ class WireRunRequest(_WireModel): gateway_policy: Optional[WireGatewayPolicy] = Field( default=None, alias="gatewayPolicy" ) + # The derived gateway-tools guidance and its prompt carrier. Its own field so the runner + # splices it at environment build and the session fingerprint can exclude it (an integration + # add must not evict a warm session). Omitted when the agent configures no connection. + gateway_guidance: Optional[WireGatewayGuidance] = Field( + default=None, alias="gatewayGuidance" + ) system_prompt: Optional[str] = Field(default=None, alias="systemPrompt") append_system_prompt: Optional[str] = Field( default=None, alias="appendSystemPrompt" diff --git a/sdks/python/agenta/sdk/utils/types.py b/sdks/python/agenta/sdk/utils/types.py index 082ae5fa72..db1aa75c11 100644 --- a/sdks/python/agenta/sdk/utils/types.py +++ b/sdks/python/agenta/sdk/utils/types.py @@ -1342,7 +1342,7 @@ class _HarnessSchema(BaseModel): """The coding agent to drive plus its execution knobs (was the flat ``harness`` scalar and its ``harness_kwargs`` slice). - ``kind`` is the harness selector (the bare ``pi_core`` / ``pi_agenta`` / ``claude`` value). + ``kind`` is the harness selector (the bare ``pi_core`` / ``claude`` / ``codex`` value). ``permissions`` is the allow/ask/deny rule lists that decide which tools may run. ``extras`` is the per-harness escape hatch (Pi's ``system`` / ``append_system`` prompt overrides).""" @@ -1352,8 +1352,7 @@ class _HarnessSchema(BaseModel): default=_DEFAULT_HARNESS, title="Harness", description=( - "Coding agent to drive: pi_core (plain Pi), claude, or pi_agenta (Pi with " - "Agenta's forced skills, tools, and base instructions)." + "Coding agent to drive: pi_core (Pi), claude (Claude Code), or codex (Codex)." ), json_schema_extra=_harness_field_schema_extra(), ) diff --git a/sdks/python/oss/tests/pytest/integration/agents/_fake_runner_backend.py b/sdks/python/oss/tests/pytest/integration/agents/_fake_runner_backend.py index fc17396835..d83f5d6977 100644 --- a/sdks/python/oss/tests/pytest/integration/agents/_fake_runner_backend.py +++ b/sdks/python/oss/tests/pytest/integration/agents/_fake_runner_backend.py @@ -124,7 +124,7 @@ class FakeRunnerBackend(Backend): """ supported_harnesses = frozenset( - {HarnessKind.PI, HarnessKind.CLAUDE, HarnessKind.AGENTA, HarnessKind.CODEX} + {HarnessKind.PI, HarnessKind.CLAUDE, HarnessKind.CODEX} ) def __init__( diff --git a/sdks/python/oss/tests/pytest/unit/agents/connections/test_capabilities.py b/sdks/python/oss/tests/pytest/unit/agents/connections/test_capabilities.py index 82cd288afb..d4b68bf63e 100644 --- a/sdks/python/oss/tests/pytest/unit/agents/connections/test_capabilities.py +++ b/sdks/python/oss/tests/pytest/unit/agents/connections/test_capabilities.py @@ -29,8 +29,8 @@ def test_claude_is_anthropic_only(): assert harness_allows_provider("claude", "OpenAI") is False # case-insensitive -def test_pi_and_agenta_reach_the_vault_providers_not_arbitrary_ones(): - for harness in ("pi_core", "pi_agenta"): +def test_pi_reaches_the_vault_providers_not_arbitrary_ones(): + for harness in ("pi_core",): # Real list, not "*": the eight vault-mapped providers are reachable... for provider in PI_VAULT_PROVIDERS: assert harness_allows_provider(harness, provider) is True @@ -38,14 +38,14 @@ def test_pi_and_agenta_reach_the_vault_providers_not_arbitrary_ones(): assert harness_allows_provider(harness, "anything-custom") is False -def test_pi_and_agenta_reach_the_openai_codex_subscription_provider(): +def test_pi_reaches_the_openai_codex_subscription_provider(): """The ChatGPT/Codex subscription provider is reachable (OAuth login, no vault key). Without this, an ``openai-codex`` model fails the agent-layer pre-resolve provider check even though the runner drives the subscription fine. ``self_managed`` is the subscription path; the provider must be allowed for that mode to ever reach the runner. """ - for harness in ("pi_core", "pi_agenta"): + for harness in ("pi_core",): for provider in PI_SUBSCRIPTION_PROVIDERS: assert harness_allows_provider(harness, provider) is True assert harness_allows_provider(harness, "openai-codex") is True @@ -77,7 +77,7 @@ def test_pi_consumes_direct_and_custom_deployment_in_v1(): # Pi now publishes `custom` (the OpenAI-compatible surface) alongside `direct` so the UI can # surface those connections. The cloud surfaces remain unconsumed in v1. The openai-only # pairing on `custom` is enforced by `harness_allows_pair`, not this per-axis list. - for harness in ("pi_core", "pi_agenta"): + for harness in ("pi_core",): assert harness_allows_deployment(harness, "direct") is True assert harness_allows_deployment(harness, "custom") is True for deployment in ("bedrock", "vertex_ai", "azure"): @@ -86,7 +86,7 @@ def test_pi_consumes_direct_and_custom_deployment_in_v1(): def test_resolved_pair_validation_matches_decision_3_table(): # Every row of design Decision 3's allowed-pairs table. - for harness in ("pi_core", "pi_agenta"): + for harness in ("pi_core",): # Pi + openai + direct/custom -> allowed. assert harness_allows_pair(harness, "openai", "direct") is True assert harness_allows_pair(harness, "openai", "custom") is True @@ -117,7 +117,7 @@ def test_claude_consumes_custom_gateway_bedrock_and_vertex(): def test_capabilities_document_shape(): doc = harness_capabilities_document() - assert set(doc) == {"pi_core", "pi_agenta", "claude", "codex"} + assert set(doc) == {"pi_core", "claude", "codex"} assert doc["claude"]["providers"] == ["anthropic"] assert doc["claude"]["model_selection"] == "alias" assert doc["pi_core"]["providers"] == list(PI_VAULT_PROVIDERS) + list( @@ -145,12 +145,11 @@ def test_capabilities_document_shape(): } } assert "mcp" not in doc["pi_core"] - assert "mcp" not in doc["pi_agenta"] def test_every_harness_publishes_a_models_map(): doc = harness_capabilities_document() - for harness in ("pi_core", "pi_agenta", "claude"): + for harness in ("pi_core", "claude"): assert isinstance(doc[harness]["models"], dict) assert doc[harness]["models"], f"{harness} has an empty models map" @@ -158,7 +157,7 @@ def test_every_harness_publishes_a_models_map(): def test_pi_models_are_a_subset_of_the_shared_catalog(): # Each Pi harness publishes, per vault provider, exactly that provider's catalog ids, plus the # subscription/OAuth providers' explicit ids (which the shared catalog does not list). - for harness in ("pi_core", "pi_agenta"): + for harness in ("pi_core",): models = HARNESS_CONNECTION_CAPABILITIES[harness].models # The published providers are the vault-mapped ones plus the subscription providers. assert set(models) == set(PI_VAULT_PROVIDERS) | set(PI_SUBSCRIPTION_PROVIDERS) @@ -177,7 +176,7 @@ def test_pi_models_are_a_subset_of_the_shared_catalog(): def test_pi_publishes_concrete_gpt_5_6_models_for_both_openai_providers(): expected = ["gpt-5.6-sol", "gpt-5.6-terra", "gpt-5.6-luna"] - for harness in ("pi_core", "pi_agenta"): + for harness in ("pi_core",): models = HARNESS_CONNECTION_CAPABILITIES[harness].models for provider in ("openai", "openai-codex"): assert models[provider][:3] == expected diff --git a/sdks/python/oss/tests/pytest/unit/agents/connections/test_model_catalog.py b/sdks/python/oss/tests/pytest/unit/agents/connections/test_model_catalog.py index 2f36b13767..22f67bc779 100644 --- a/sdks/python/oss/tests/pytest/unit/agents/connections/test_model_catalog.py +++ b/sdks/python/oss/tests/pytest/unit/agents/connections/test_model_catalog.py @@ -31,7 +31,7 @@ pi_model_catalog, ) -_ALL_HARNESSES = ("pi_core", "pi_agenta", "claude", "codex") +_ALL_HARNESSES = ("pi_core", "claude", "codex") def test_data_files_load_and_validate(): @@ -255,7 +255,7 @@ def test_model_catalog_entries_helper_matches_the_published_field(): assert model_catalog_entries("some-future-harness") == [] -@pytest.mark.parametrize("harness", ["pi_core", "pi_agenta"]) +@pytest.mark.parametrize("harness", ["pi_core"]) def test_pi_input_modalities_lookup_joins_resolved_provider_and_model(harness): assert model_input_modalities(harness, "gpt-5.5", provider="openai") == [ "text", diff --git a/sdks/python/oss/tests/pytest/unit/agents/golden/run_request.gateway_connection.json b/sdks/python/oss/tests/pytest/unit/agents/golden/run_request.gateway_connection.json index 76709eb066..d5d2774a56 100644 --- a/sdks/python/oss/tests/pytest/unit/agents/golden/run_request.gateway_connection.json +++ b/sdks/python/oss/tests/pytest/unit/agents/golden/run_request.gateway_connection.json @@ -24,7 +24,7 @@ "customTools": [ { "name": "search_tools", - "description": "Find tools across the integrations connected to this agent. Describe the task you want to perform; the result carries the integration, the tool key, and the input schema to call it with. Returns at most 5 results, best matches first \u2014 a cap, not the whole catalog. If the search fails, retry it once and no more. If nothing matched, search again with a more specific description of the task, then stop. Never invent an integration name. Connected integrations: github.", + "description": "Find tools across the integrations connected to this agent. Describe the task you want to perform; the result carries the integration, the tool key, and the input schema to call it with. Returns at most 5 results, best matches first \u2014 a cap, not the whole catalog. If the search fails, retry it once and no more. If nothing matched, search again with a more specific description of the task, then stop. Never invent an integration name.", "inputSchema": { "type": "object", "properties": { @@ -78,6 +78,10 @@ "permissions": { "default": "allow_reads" }, + "gatewayGuidance": { + "text": "## Connected integrations\n\nYou can reach your integrations with two tools: `search_tools` and `run_tool`.\nFor instance, some of the integrations you have: github. Others may exist, and this\nlist can go stale \u2014 `search_tools` is the source of truth for what is connected right now.\n\n- Search once per task, with a concrete description of what you want to do. Never repeat an\n equivalent query \u2014 a second search that means the same thing returns the same results.\n- A search returns at most 5 results. That is a cap, not the whole catalog \u2014 if none fit,\n narrow the description rather than concluding no such tool exists.\n- \"No configured tool matched this request.\" is not a failure. Refine the query ONCE and\n search again \u2014 that is what the message asks for \u2014 then report if it still finds nothing.\n- \"Tool search is temporarily unavailable.\" is a temporary failure: retry it once and no more.\n- Use only an integration and a tool key that a search result returned. Never invent one.\n Pass the BARE tool key, not a prefixed provider action id such as `GMAIL_FETCH_EMAILS`.\n- Copy the arguments from the input schema the search result returned.\n- Stop searching once a result is usable, and run it.\n- A run may pause for the user's approval or be refused outright: that is this agent's\n permission policy, not a bug. A refusal will not succeed on a retry or with reshaped\n arguments \u2014 report it instead of looping.", + "carrier": "appendSystemPrompt" + }, "gatewayPolicy": { "integrations": { "github": { diff --git a/sdks/python/oss/tests/pytest/unit/agents/platform/test_gateway_connection_resolve.py b/sdks/python/oss/tests/pytest/unit/agents/platform/test_gateway_connection_resolve.py index 08533ba84b..ea7acd38a1 100644 --- a/sdks/python/oss/tests/pytest/unit/agents/platform/test_gateway_connection_resolve.py +++ b/sdks/python/oss/tests/pytest/unit/agents/platform/test_gateway_connection_resolve.py @@ -215,27 +215,28 @@ async def test_search_tools_names_the_connected_integrations(fake_http, connecti ) search = next(s for s in resolution.tool_specs if s.name == "search_tools") - # Sorted, so the same agent always presents the same sentence. - assert search.description.endswith("Connected integrations: github, slack.") - # Names only. A slug is not the model's to know, and `run_tool` names no integration - # because it is told which one in the arguments. + # The descriptions are STABLE across integration sets: names live in the gatewayGuidance + # prompt section, because `customTools` is fingerprinted and a names sentence here made + # adding an integration evict the warm session. + assert "Connected integrations" not in search.description + # A slug is never the model's to know, on any surface. for slug in ["github-work", "slack-main"]: assert slug not in search.description run = next(s for s in resolution.tool_specs if s.name == "run_tool") assert "Connected integrations" not in run.description -def test_an_empty_integration_list_leaves_no_dangling_sentence(): - """The resolver's caller guards this, so it is only reachable by a future one. - - Worth being total about: the failure mode is a malformed sentence in model-facing - text, which no type or test elsewhere would catch. - """ +def test_derived_specs_are_identical_for_any_integration_set(): + """The stability guarantee itself: the two derived tools are byte-identical whether the + agent has zero, one, or many integrations, so only the FIRST connection (which adds the + tools) can change session config.""" from agenta.sdk.agents.platform.gateway import _derived_tool_specs - search = next(s for s in _derived_tool_specs([]) if s.name == "search_tools") + none = _derived_tool_specs([]) + many = _derived_tool_specs(["github", "slack", "linear"]) - assert "Connected integrations" not in search.description + assert [s.model_dump() for s in none] == [s.model_dump() for s in many] + search = next(s for s in none if s.name == "search_tools") assert search.description.endswith("Never invent an integration name.") diff --git a/sdks/python/oss/tests/pytest/unit/agents/test_agent_composition_seam.py b/sdks/python/oss/tests/pytest/unit/agents/test_agent_composition_seam.py index 84bf67938e..d8171e326c 100644 --- a/sdks/python/oss/tests/pytest/unit/agents/test_agent_composition_seam.py +++ b/sdks/python/oss/tests/pytest/unit/agents/test_agent_composition_seam.py @@ -12,7 +12,12 @@ import pytest -from agenta.sdk.agents import AgentResult, HarnessKind, Message +from agenta.sdk.agents import ( + AgentResult, + HarnessKind, + InvalidHarnessKindError, + Message, +) from agenta.sdk.agents.connections import ( ConnectionResolutionError, ResolvedConnection, @@ -83,9 +88,7 @@ async def destroy(self) -> None: class _FakeBackend(Backend): - supported_harnesses = frozenset( - {HarnessKind.PI, HarnessKind.CLAUDE, HarnessKind.AGENTA} - ) + supported_harnesses = frozenset({HarnessKind.PI, HarnessKind.CLAUDE}) def __init__(self, *, output: str = "hi") -> None: self._output = output @@ -385,6 +388,66 @@ async def _resolve(*, model, context): ) +async def test_a_config_persisted_with_an_unreadable_harness_is_refused_with_a_shape(): + """F4: a config stored before the commit boundary existed must still fail with a code. + + The gate's H1 cell persisted `harness.kind` as `12345` and the invoke that followed died on + the enum's bare `ValueError`, which the remap turned into a 500 whose body was the Python + repr. The refusal now happens where the handler reads the template, before it selects a + backend or resolves anything, and it carries the field, the value, and the harnesses that + exist. + """ + backend = _FakeBackend() + + async def _must_not_run(*, model, context): + raise AssertionError("resolution must not run on an unreadable harness") + + comp = AgentComposition( + select_backend=lambda template: backend, + resolve_connection=_must_not_run, + ) + handler = make_agent_handler(comp) + + with pytest.raises(InvalidHarnessKindError) as caught: + await handler( + request=_request(), + messages=[{"role": "user", "content": "hi"}], + parameters=_params(12345, model={"provider": "openai", "model": "gpt-5.5"}), + ) + + assert caught.value.code == 400 + assert "harness.kind" in caught.value.message + # Nothing ran: no session was created, so no turn can be stored for a config that cannot run. + assert backend.created_configs == [] + + +async def test_a_run_configured_with_the_harness_enum_itself_actually_runs(): + """The SDK's own `HarnessKind` member must be usable as input, end to end. + + `HarnessKind` is a `str` Enum, so `str(member)` is "HarnessKind.CLAUDE". Stringifying the + caller's value lower-cased that to "harnesskind.claude", which `make_harness` then refused — + valid input, rejected by the parser that was supposed to accept it. + """ + backend = _FakeBackend(output="hi") + comp = AgentComposition( + select_backend=lambda template: backend, + resolve_connection=_no_connection, + ) + handler = make_agent_handler(comp) + + await handler( + request=_request(), + messages=[{"role": "user", "content": "hi"}], + parameters=_params(HarnessKind.CLAUDE), + ) + + # It reached the backend, which is what a mangled value never did. + assert len(backend.created_configs) == 1 + assert backend.created_effective_parameters[0]["agent"]["harness"]["kind"] == ( + HarnessKind.CLAUDE + ) + + async def test_composition_override_replaces_default_gating(): """A composition MAY still fully replace resolve_session_connection (bare passthrough), proving the seam stays injectable rather than hardcoding the gate.""" diff --git a/sdks/python/oss/tests/pytest/unit/agents/test_dtos_capabilities_events.py b/sdks/python/oss/tests/pytest/unit/agents/test_dtos_capabilities_events.py index 0bf431aebb..55ed542597 100644 --- a/sdks/python/oss/tests/pytest/unit/agents/test_dtos_capabilities_events.py +++ b/sdks/python/oss/tests/pytest/unit/agents/test_dtos_capabilities_events.py @@ -81,7 +81,8 @@ def test_harness_type_coerce(): assert HarnessKind.coerce(HarnessKind.PI) is HarnessKind.PI assert HarnessKind.coerce("pi_core") is HarnessKind.PI assert HarnessKind.coerce("PI_CORE") is HarnessKind.PI # case-insensitive - assert HarnessKind.coerce("pi_agenta") is HarnessKind.AGENTA + # The removed ``pi_agenta`` experiment's spelling still reads as Pi (old stored revisions). + assert HarnessKind.coerce("pi_agenta") is HarnessKind.PI assert HarnessKind.coerce("claude") is HarnessKind.CLAUDE with pytest.raises(ValueError): HarnessKind.coerce("bogus") diff --git a/sdks/python/oss/tests/pytest/unit/agents/test_dtos_harness_kind_refusal.py b/sdks/python/oss/tests/pytest/unit/agents/test_dtos_harness_kind_refusal.py new file mode 100644 index 0000000000..43e3ebcb2a --- /dev/null +++ b/sdks/python/oss/tests/pytest/unit/agents/test_dtos_harness_kind_refusal.py @@ -0,0 +1,132 @@ +"""F4: a harness kind the runtime cannot read must be refused with a shape, at every boundary. + +The gate's H1 cell committed `harness.kind` as `12345` and as `"not_a_real_harness"`. Both were +persisted with a 200, and the invoke that followed died on the enum's bare `ValueError`: an +unhandled HTTP 500 whose body was the Python repr, carrying no code a client could act on and +no hint of which field was wrong. + +These cases pin the SDK half: `coerce` raises a typed, coded error; parsing a template refuses a +bad kind where the template is read; and the invoke remap turns it into a 400 naming the field +and the harnesses that exist, never a 500. An absent or null kind still means "use the default", +which is what every stored config relies on. + +Run: uv run pytest oss/tests/pytest/unit/agents/test_dtos_harness_kind_refusal.py +""" + +from __future__ import annotations + +import json + +import pytest + +from agenta.sdk.agents.dtos import ( + AgentTemplate, + HarnessKind, + InvalidHarnessKindError, +) +from agenta.sdk.decorators.routing import handle_invoke_failure + + +def _params(kind): + return {"agent": {"harness": {"kind": kind}}} + + +class TestCoerce: + @pytest.mark.parametrize("value", [12345, "not_a_real_harness", 0, [], {}]) + def test_an_unreadable_kind_raises_the_coded_error(self, value): + with pytest.raises(InvalidHarnessKindError) as caught: + HarnessKind.coerce(value) + + assert caught.value.code == 400 + assert "harness.kind" in caught.value.message + + def test_the_message_names_the_value_and_every_harness_that_exists(self): + with pytest.raises(InvalidHarnessKindError) as caught: + HarnessKind.coerce("not_a_real_harness") + + message = caught.value.message + assert "not_a_real_harness" in message + for kind in HarnessKind: + assert kind.value in message + + def test_it_is_still_a_value_error(self): + # The enum has always raised `ValueError` here, and callers guard on that type. The + # coded error inherits it so those guards keep working. + with pytest.raises(ValueError): + HarnessKind.coerce("not_a_real_harness") + + @pytest.mark.parametrize( + "value,expected", + [ + ("pi_core", HarnessKind.PI), + ("PI_CORE", HarnessKind.PI), + ("claude", HarnessKind.CLAUDE), + ("codex", HarnessKind.CODEX), + (HarnessKind.CLAUDE, HarnessKind.CLAUDE), + # A revision saved while the experiment existed still reads as plain Pi. + ("pi_agenta", HarnessKind.PI), + ], + ) + def test_a_readable_kind_is_unchanged(self, value, expected): + assert HarnessKind.coerce(value) is expected + + +class TestTemplateParsing: + @pytest.mark.parametrize("kind", [12345, "not_a_real_harness", 0]) + def test_a_bad_kind_is_refused_where_the_template_is_read(self, kind): + # This is the invoke boundary: the handler parses the template before it selects a + # backend, so a config persisted before the fix fails here rather than at `make_harness`. + with pytest.raises(InvalidHarnessKindError): + AgentTemplate.from_params(_params(kind)) + + @pytest.mark.parametrize("kind", [None, "", " "]) + def test_an_absent_kind_still_means_the_default(self, kind): + # Unchanged behaviour, deliberately: null is how a caller says "whatever the default + # is", and refusing it would break every config that never set a harness. + assert ( + AgentTemplate.from_params(_params(kind)).harness == AgentTemplate().harness + ) + + def test_a_template_with_no_harness_section_is_unchanged(self): + assert ( + AgentTemplate.from_params({"agent": {"instructions": "hi"}}).harness + == AgentTemplate().harness + ) + + @pytest.mark.parametrize("kind", ["pi_core", "claude", "codex"]) + def test_a_readable_kind_parses_to_itself(self, kind): + assert AgentTemplate.from_params(_params(kind)).harness == kind + + def test_a_legacy_pi_agenta_revision_still_parses(self): + # Its stored spelling survives the parse exactly as before; `make_harness` maps it. + assert AgentTemplate.from_params(_params("pi_agenta")).harness == "pi_agenta" + + @pytest.mark.parametrize("member", list(HarnessKind)) + def test_the_sdks_own_enum_is_usable_as_input(self, member): + # `HarnessKind` is a `str` Enum, so `str(member)` is "HarnessKind.CLAUDE", not "claude". + # Stringifying the caller's value lower-cased that to "harnesskind.claude", which then + # failed at `make_harness` — the SDK's own enum was not accepted by the SDK's parser. + assert AgentTemplate.from_params(_params(member)).harness == member.value + + def test_a_member_survives_the_round_trip_back_to_a_harness(self): + # The parse and the lookup have to agree: whatever `from_params` produces must be a + # value `coerce` reads back, which is the hop that used to break. + parsed = AgentTemplate.from_params(_params(HarnessKind.CLAUDE)).harness + assert HarnessKind.coerce(parsed) is HarnessKind.CLAUDE + + +class TestInvokeRemap: + async def test_it_answers_400_with_the_field_and_the_allowed_values(self): + response = await handle_invoke_failure(InvalidHarnessKindError("not_a_harness")) + body = json.dumps(json.loads(bytes(response.body))) + + assert response.status_code == 400 + assert "harness.kind" in body + assert "invalid-harness-kind" in body + + async def test_the_bare_value_error_it_replaced_would_have_been_a_500(self): + # The shape of the defect: an unclassified exception is a 500 whose body is the repr. + response = await handle_invoke_failure( + ValueError("'12345' is not a valid HarnessKind") + ) + assert response.status_code == 500 diff --git a/sdks/python/oss/tests/pytest/unit/agents/test_harness_adapters.py b/sdks/python/oss/tests/pytest/unit/agents/test_harness_adapters.py index d25fc67779..b3515f7b82 100644 --- a/sdks/python/oss/tests/pytest/unit/agents/test_harness_adapters.py +++ b/sdks/python/oss/tests/pytest/unit/agents/test_harness_adapters.py @@ -13,8 +13,6 @@ import pytest from agenta.sdk.agents import ( - AgentaAgentTemplate, - AgentaHarness, AgentTemplate, ClaudeAgentTemplate, ClaudeHarness, @@ -29,14 +27,7 @@ UnsupportedHarnessError, make_harness, ) -from agenta.sdk.agents.adapters.agenta_builtins import ( - AGENTA_FORCED_APPEND_SYSTEM, - AGENTA_FORCED_SKILLS, - GETTING_STARTED_WITH_AGENTA_SKILL, - AGENTA_PREAMBLE, - force_skills, - gateway_guidance, -) +from agenta.sdk.agents.adapters.agenta_builtins import gateway_guidance from agenta.sdk.agents.adapters.harnesses import _normalize_tool_specs, _opt_str from agenta.sdk.agents.tools import ( CompiledTool, @@ -97,24 +88,6 @@ def test_pi_threads_model_ref_so_connection_reaches_resolver(make_env): assert result.wire_model_connection() == {} -def test_agenta_threads_model_ref_so_connection_reaches_resolver(make_env): - """Same guarantee as Pi for the ``pi_agenta`` harness (it also runs Pi).""" - harness = AgentaHarness(make_env(supported=[HarnessKind.AGENTA])) - agent = AgentTemplate( - instructions="hi", - model={ - "model": "gpt-4o-mini", - "connection": {"mode": "agenta", "slug": "my-compat"}, - }, - ) - - result = harness._to_harness_config(_session_config(agent=agent)) - - assert result.model_ref is not None - assert result.model_ref.connection.slug == "my-compat" - assert result.wire_model_connection() == {} - - def test_pi_reads_its_harness_extras_slice(make_env): harness = PiHarness(make_env(supported=[HarnessKind.PI])) agent = AgentTemplate( @@ -148,110 +121,6 @@ def test_pi_drops_blank_harness_extras(make_env): assert result.wire_prompt() == {} -# ------------------------------------------------------------------------- Agenta - - -def test_agenta_forces_preamble_and_persona_and_carries_skills(make_env): - harness = AgentaHarness(make_env(supported=[HarnessKind.AGENTA])) - skill = { - "name": "release-notes", - "description": "Draft release notes.", - "body": "Read the changelog, then write notes.", - } - config = _session_config( - agent=AgentTemplate( - instructions="My project rules.", model="m", skills=[skill] - ), - custom_tools=[{"name": "t", "callRef": "ref"}], - tool_callback=_CALLBACK, - ) - - result = harness._to_harness_config(config) - - assert isinstance(result, AgentaAgentTemplate) - # AGENTS.md is the base preamble with the author's instructions appended after it. - assert result.agents_md.startswith(AGENTA_PREAMBLE) - assert result.agents_md.endswith("My project rules.") - # The author's resolved inline skills ride the config, plus the forced platform skill(s) the - # harness always injects. The author's skill comes first; the platform skill is appended. - skill_names = [s.name for s in result.skills] - assert skill_names[0] == "release-notes" - assert GETTING_STARTED_WITH_AGENTA_SKILL.name in skill_names - assert "skills" not in result.wire_tools() - assert result.wire_skills()["skills"][0]["name"] == "release-notes" - # The persona is forced onto append_system; custom tools and callback pass through. - assert result.append_system.startswith(AGENTA_FORCED_APPEND_SYSTEM) - assert result.custom_tools[0]["name"] == "t" - assert result.tool_callback is _CALLBACK - - -def test_agenta_forces_platform_skill_on_a_skill_less_config(make_env): - # The actually-forced behavior: a custom pi_agenta config with NO skills (the default - # template's `_agenta` embed dropped) still carries the platform skill on every run. - harness = AgentaHarness(make_env(supported=[HarnessKind.AGENTA])) - config = _session_config( - agent=AgentTemplate(instructions="My project rules.", model="m", skills=[]) - ) - - result = harness._to_harness_config(config) - - assert [s.name for s in result.skills] == [GETTING_STARTED_WITH_AGENTA_SKILL.name] - - -def test_agenta_does_not_duplicate_an_already_present_platform_skill(make_env): - # A config that already carries the resolved platform skill (e.g. via the default template's - # embed) is not doubled: the author's copy wins on the name clash. - harness = AgentaHarness(make_env(supported=[HarnessKind.AGENTA])) - existing = GETTING_STARTED_WITH_AGENTA_SKILL.model_dump(mode="json") - config = _session_config( - agent=AgentTemplate(instructions="hi", model="m", skills=[existing]) - ) - - result = harness._to_harness_config(config) - - names = [s.name for s in result.skills] - assert names.count(GETTING_STARTED_WITH_AGENTA_SKILL.name) == 1 - - -def test_force_skills_unions_forced_after_author_skills(): - from agenta.sdk.agents.skills import SkillTemplate - - author = SkillTemplate( - name="release-notes", description="Draft notes.", body="Do it." - ) - - out = force_skills([author]) - - assert out[0] is author - assert {s.name for s in out} == {"release-notes"} | { - s.name for s in AGENTA_FORCED_SKILLS - } - - -def test_agenta_passes_through_user_pi_options(make_env): - harness = AgentaHarness(make_env(supported=[HarnessKind.AGENTA])) - agent = AgentTemplate( - instructions="hi", - harness_extras={"system": "You are Pi.", "append_system": "Be terse."}, - ) - - result = harness._to_harness_config(_session_config(agent=agent)) - - # `system` passes through; the author's `append_system` is appended after the forced persona. - assert result.system == "You are Pi." - assert result.append_system.startswith(AGENTA_FORCED_APPEND_SYSTEM) - assert result.append_system.endswith("Be terse.") - - -def test_agenta_is_sandbox_agent_supported(): - # Agenta is Pi with an opinion, so the sandbox-agent backend drives it too (on the `pi` ACP - # agent, with the runner laying the forced skills into the sandbox). This is what lets - # `agenta` run on a non-local sandbox (e.g. daytona) instead of raising. - from agenta.sdk.agents import SandboxAgentBackend - - assert SandboxAgentBackend(url="http://runner").supports(HarnessKind.AGENTA) - - # ------------------------------------------------------------------------- Claude @@ -415,7 +284,6 @@ def test_make_harness_maps_string_to_class(make_env): HarnessKind.PI, HarnessKind.CLAUDE, HarnessKind.CODEX, - HarnessKind.AGENTA, ] ) assert isinstance(make_harness("pi_core", env), PiHarness) @@ -426,8 +294,8 @@ def test_make_harness_maps_string_to_class(make_env): assert isinstance(make_harness(HarnessKind.CLAUDE, env), ClaudeHarness) assert isinstance(make_harness("codex", env), CodexHarness) assert isinstance(make_harness(HarnessKind.CODEX, env), CodexHarness) - assert isinstance(make_harness("pi_agenta", env), AgentaHarness) - assert isinstance(make_harness(HarnessKind.AGENTA, env), AgentaHarness) + # The removed experiment's spelling still resolves (to plain Pi) for old stored configs. + assert isinstance(make_harness("pi_agenta", env), PiHarness) # ------------------------------------------------- gateway connection prompt guidance @@ -456,10 +324,9 @@ def test_make_harness_maps_string_to_class(make_env): _AUTHOR_APPEND = "Be terse." _AUTHOR_INSTRUCTIONS = "My project rules." _HARNESS_CASES = [ - (PiHarness, HarnessKind.PI, "append_system", _AUTHOR_APPEND), - (ClaudeHarness, HarnessKind.CLAUDE, "agents_md", _AUTHOR_INSTRUCTIONS), - (CodexHarness, HarnessKind.CODEX, "agents_md", _AUTHOR_INSTRUCTIONS), - (AgentaHarness, HarnessKind.AGENTA, "agents_md", _AUTHOR_INSTRUCTIONS), + (PiHarness, HarnessKind.PI, "append_system", "appendSystemPrompt", _AUTHOR_APPEND), + (ClaudeHarness, HarnessKind.CLAUDE, "agents_md", "agentsMd", _AUTHOR_INSTRUCTIONS), + (CodexHarness, HarnessKind.CODEX, "agents_md", "agentsMd", _AUTHOR_INSTRUCTIONS), ] @@ -471,7 +338,9 @@ def test_every_registered_harness_declares_a_guidance_carrier(): """ from agenta.sdk.agents.adapters.harnesses import _HARNESSES - assert {kind for _cls, kind, _carrier, _author in _HARNESS_CASES} == set(_HARNESSES) + assert {kind for _cls, kind, _field, _carrier, _author in _HARNESS_CASES} == set( + _HARNESSES + ) def _guidance_agent() -> AgentTemplate: @@ -482,33 +351,43 @@ def _guidance_agent() -> AgentTemplate: ) -@pytest.mark.parametrize("harness_cls,kind,carrier,author_text", _HARNESS_CASES) +@pytest.mark.parametrize( + "harness_cls,kind,prompt_field,carrier,author_text", _HARNESS_CASES +) def test_gateway_guidance_reaches_every_harness( - make_env, harness_cls, kind, carrier, author_text + make_env, harness_cls, kind, prompt_field, carrier, author_text ): """Every harness gets the same two derived tools, so every one gets the instructions. - A section added to the Agenta preamble alone would leave Pi, Claude, and Codex holding - `search_tools` and `run_tool` with nothing telling them how to use them. + The guidance now rides its own `gatewayGuidance` wire field (spliced by the runner at + environment build, so integration adds stop evicting warm sessions); the adapter's job + is to emit the field with the right carrier and keep the prompt strings purely authored. """ harness = harness_cls(make_env(supported=[kind])) config = _session_config(agent=_guidance_agent(), gateway_policy=_GATEWAY_POLICY) result = harness._to_harness_config(config) - text = getattr(result, carrier) - assert "search_tools" in text - assert "run_tool" in text - # The configured integration names, and only those. - assert "github, slack" in text - # The author's own text on that layer still comes last: the guidance is the platform half. - assert text.endswith(author_text) - assert text.index("search_tools") < text.index(author_text) + guidance = result.gateway_guidance + assert guidance is not None + assert guidance.carrier == carrier + assert "search_tools" in guidance.text + assert "run_tool" in guidance.text + # The configured integration names read as EXAMPLES, so a stale list stays honest. + assert "github, slack" in guidance.text + assert "Others may exist" in guidance.text + # The prompt strings stay purely authored: the runner does the splice, not the adapter. + assert getattr(result, prompt_field) == author_text + assert result.wire_gateway_guidance() == { + "gatewayGuidance": {"text": guidance.text, "carrier": carrier} + } -@pytest.mark.parametrize("harness_cls,kind,carrier,author_text", _HARNESS_CASES) +@pytest.mark.parametrize( + "harness_cls,kind,prompt_field,carrier,author_text", _HARNESS_CASES +) def test_no_gateway_guidance_without_a_connection( - make_env, harness_cls, kind, carrier, author_text + make_env, harness_cls, kind, prompt_field, carrier, author_text ): """G6's prompt half: an agent with no connection entry gets no gateway section.""" harness = harness_cls(make_env(supported=[kind])) @@ -516,41 +395,25 @@ def test_no_gateway_guidance_without_a_connection( result = harness._to_harness_config(config) - text = getattr(result, carrier) or "" + assert result.gateway_guidance is None + assert result.wire_gateway_guidance() == {} + text = getattr(result, prompt_field) or "" assert "search_tools" not in text - assert "run_tool" not in text - -def test_pi_agents_md_stays_purely_authored(make_env): - """Pi's carrier is ``append_system``; AGENTS.md keeps only what the author wrote. - AGENTS.md is the project-conventions layer and a bare Pi run has no platform half of it, - so injecting there would put platform text in a file the author owns outright. - """ +def test_pi_prompt_strings_stay_purely_authored(make_env): + """Pi's guidance carrier is ``appendSystemPrompt``, but the SDK no longer splices it: + both prompt strings leave the adapter exactly as the author wrote them, and the field + carries the platform half for the runner to splice at build time.""" harness = PiHarness(make_env(supported=[HarnessKind.PI])) config = _session_config(agent=_guidance_agent(), gateway_policy=_GATEWAY_POLICY) result = harness._to_harness_config(config) assert result.agents_md == _AUTHOR_INSTRUCTIONS - assert "search_tools" in result.append_system - - -def test_agenta_keeps_its_preamble_first_with_guidance(make_env): - """The existing prefix rule survives: preamble, then guidance, then the author.""" - harness = AgentaHarness(make_env(supported=[HarnessKind.AGENTA])) - config = _session_config( - agent=AgentTemplate(instructions="My project rules.", model="m"), - gateway_policy=_GATEWAY_POLICY, - ) - - result = harness._to_harness_config(config) - - assert result.agents_md.startswith(AGENTA_PREAMBLE) - assert result.agents_md.endswith("My project rules.") - assert result.agents_md.index(AGENTA_PREAMBLE) < result.agents_md.index( - "search_tools" - ) + assert result.append_system == _AUTHOR_APPEND + assert result.gateway_guidance is not None + assert result.gateway_guidance.carrier == "appendSystemPrompt" def test_gateway_guidance_is_never_stored_in_the_revision(make_env): @@ -561,7 +424,8 @@ def test_gateway_guidance_is_never_stored_in_the_revision(make_env): result = harness._to_harness_config(config) - assert "search_tools" in result.append_system + assert result.gateway_guidance is not None + assert "search_tools" in result.gateway_guidance.text assert agent.instructions == _AUTHOR_INSTRUCTIONS assert agent.harness_extras == {"append_system": _AUTHOR_APPEND} @@ -575,8 +439,9 @@ def test_gateway_guidance_carries_all_six_prompt_items(): """ section = gateway_guidance(["github", "slack"]) - # 1. The configured integration names. - assert "Configured integrations: github, slack." in section + # 1. The configured integration names, framed as examples so a stale list stays honest. + assert "some of the integrations you have: github, slack" in section + assert "Others may exist" in section # 2. Search once per task, with a concrete description, and no equivalent repeats. assert "Search once per task, with a concrete description" in section assert "Never repeat an\n equivalent query" in section @@ -606,9 +471,11 @@ def test_gateway_guidance_is_absent_for_an_empty_policy(): assert gateway_guidance([]) is None -@pytest.mark.parametrize("harness_cls,kind,carrier,author_text", _HARNESS_CASES) +@pytest.mark.parametrize( + "harness_cls,kind,prompt_field,carrier,author_text", _HARNESS_CASES +) def test_gateway_policy_stays_out_of_every_harness_config( - make_env, harness_cls, kind, carrier, author_text + make_env, harness_cls, kind, prompt_field, carrier, author_text ): """Runner policy stays neutral while harnesses receive only names for guidance.""" harness = harness_cls(make_env(supported=[kind])) diff --git a/sdks/python/oss/tests/pytest/unit/agents/test_harness_identity.py b/sdks/python/oss/tests/pytest/unit/agents/test_harness_identity.py index ab99fdb7dc..4e5a4e270a 100644 --- a/sdks/python/oss/tests/pytest/unit/agents/test_harness_identity.py +++ b/sdks/python/oss/tests/pytest/unit/agents/test_harness_identity.py @@ -31,7 +31,7 @@ def test_identity_value_is_the_bare_harness_string(): # The identity's `value` is the bare HarnessKind value (the runtime/wire selector), NOT the # slug — so the wire/runner contract is unchanged. values = {identity.value for identity in HARNESS_IDENTITIES} - assert values == {"pi_core", "pi_agenta", "claude", "codex"} + assert values == {"pi_core", "claude", "codex"} def _harness_kind_field(): diff --git a/sdks/python/oss/tests/pytest/unit/agents/test_wire_contract.py b/sdks/python/oss/tests/pytest/unit/agents/test_wire_contract.py index 13f1d1f52a..9c59aa0df8 100644 --- a/sdks/python/oss/tests/pytest/unit/agents/test_wire_contract.py +++ b/sdks/python/oss/tests/pytest/unit/agents/test_wire_contract.py @@ -10,7 +10,7 @@ to match. There is no engine selector on the wire: the runner drives one engine (the sandbox-agent ACP -path) and ``harness`` (``pi_core`` / ``pi_agenta`` / ``claude``) picks the agent. +path) and ``harness`` (``pi_core`` / ``claude`` / ``codex``) picks the agent. """ from __future__ import annotations @@ -23,7 +23,6 @@ from agenta.sdk.redaction.redactor import Redactor from agenta.sdk.agents import ( - AgentaAgentTemplate, AgentTemplate, ClaudeAgentTemplate, CodexAgentTemplate, @@ -46,6 +45,7 @@ ToolResolver, TraceContext, ) +from agenta.sdk.agents.adapters.agenta_builtins import gateway_guidance_field from agenta.sdk.agents.platform.gateway import _derived_tool_specs from agenta.sdk.agents.tools import ( CompiledTool, @@ -93,6 +93,7 @@ "toolCallback", "permissions", "gatewayPolicy", + "gatewayGuidance", "systemPrompt", "appendSystemPrompt", "skills", @@ -322,6 +323,11 @@ def _gateway_connection_payload(): # Straight from the producer, so the golden records what a real resolve emits. custom_tools=_derived_tool_specs(list(gateway_policy.integrations)), tool_callback=_CALLBACK, + # Straight from the same producer path the adapters use, so the golden pins the + # separate-field form (the guidance is spliced runner-side at environment build). + gateway_guidance=gateway_guidance_field( + list(gateway_policy.integrations), "appendSystemPrompt" + ), ) return request_to_wire( harness=HarnessKind.PI, @@ -334,23 +340,6 @@ def _gateway_connection_payload(): ) -def _agenta_payload(): - config = AgentaAgentTemplate( - agents_md="Agenta preamble + project rules.", - model="gpt-5.5", - custom_tools=[dict(_CUSTOM_TOOL)], - tool_callback=_CALLBACK, - append_system="You are an Agenta agent.", - skills=[dict(_SKILL)], - ) - return request_to_wire( - harness=HarnessKind.AGENTA, - sandbox="local", - config=config, - messages=[Message(role="user", content="hi")], - ) - - def _attachment_payload(): config = PiAgentTemplate( agents_md="Use the attached file.", @@ -433,18 +422,6 @@ def test_request_to_wire_omits_gateway_policy_without_a_connection(golden): assert payload == golden(name) -def test_request_to_wire_agenta_carries_skills_and_pi_shape(): - payload = _agenta_payload() - assert set(payload) <= KNOWN_REQUEST_KEYS - # Agenta is a Pi config: same tool shape and shared permission plan, plus prompt overrides. - assert payload["permissions"] == {"default": "allow_reads"} - assert payload["tools"] == list(PI_BUILTIN_TOOL_NAMES) - assert payload["appendSystemPrompt"] == "You are an Agenta agent." - # ...plus the resolved inline skill packages, on their own seam (not in `wire_tools`). - assert payload["skills"][0]["name"] == "release-notes" - assert payload["skills"][0]["files"][0]["path"] == "scripts/draft.py" - - def test_request_to_wire_skills_ride_their_own_seam_not_tools(): # Skills are emitted by `wire_skills`, not folded into the tool wire. config = PiAgentTemplate(skills=[dict(_SKILL)]) diff --git a/sdks/python/oss/tests/pytest/unit/agents/test_wire_models.py b/sdks/python/oss/tests/pytest/unit/agents/test_wire_models.py index 4c7eeb9d31..48c8bb3152 100644 --- a/sdks/python/oss/tests/pytest/unit/agents/test_wire_models.py +++ b/sdks/python/oss/tests/pytest/unit/agents/test_wire_models.py @@ -36,7 +36,6 @@ from .test_wire_contract import ( KNOWN_REQUEST_KEYS, - _agenta_payload, _claude_payload, _pi_payload, ) @@ -113,7 +112,7 @@ def test_goldens_validate_against_the_exported_schema(golden, golden_name, ag_ty def test_request_to_wire_output_validates_against_the_schema(): # The producer and the schema agree: the dict `request_to_wire` builds for each harness # validates against the exported request schema and round-trips through the wire model. - for payload in (_pi_payload(), _claude_payload(), _agenta_payload()): + for payload in (_pi_payload(), _claude_payload()): jsonschema.validate(payload, CATALOG_TYPES["run_request"]) WireRunRequest.model_validate(payload) diff --git a/sdks/python/oss/tests/pytest/unit/agents/tools/test_subagent_tool_name.py b/sdks/python/oss/tests/pytest/unit/agents/tools/test_subagent_tool_name.py new file mode 100644 index 0000000000..514adee6f4 --- /dev/null +++ b/sdks/python/oss/tests/pytest/unit/agents/tools/test_subagent_tool_name.py @@ -0,0 +1,286 @@ +"""Subagent tool names must satisfy the provider's tool-name pattern (E4). + +THE BUG. A subagent's model-visible name is a DISPLAY name the user typed in the Subagents UI, and +it reached the provider verbatim. Every major provider requires `^[a-zA-Z0-9_.-]+$` for a tool +name and refuses the WHOLE `tools` array when any entry violates it, so adding a child called +"QA-v0.114.4 Helper" made every run of the parent fail with `Invalid 'tools[23].name'` until the +child was renamed. "Support Router" is an ordinary thing to type. + +The derivation itself is old — `ReferenceToolConfig.tool_name` has returned `self.name or +self.slug` since 2026-06-26 — but nothing put an authored display name in front of it until the +Subagents UI shipped, so the latent break became reachable. + +Two properties are load-bearing beyond "it is valid now": + + STABILITY. The model sees this name. A name that changed between turns would strand a + conversation mid-tool-call, so the mapping is deterministic and the collision discriminator is + derived from the tool's own identity rather than its position in the list. + + DISTINCTNESS. Sanitizing can merge two different children onto one name, and a duplicate tool + name silently SHADOWS the earlier tool rather than erroring — the second subagent would simply + never be callable, with no message anywhere. +""" + +from __future__ import annotations + +import re + +import pytest + +from agenta.sdk.agents.tools import ( + ReferenceToolConfig, + disambiguate_tool_names, + sanitize_tool_name, +) + +#: The pattern the providers enforce. Every name this module produces must match it. +PROVIDER_TOOL_NAME = re.compile(r"^[a-zA-Z0-9_.-]+$") + + +@pytest.mark.parametrize( + "display_name", + [ + "QA-v0.114.4 Helper", # the live repro + "Support Router", # the ordinary case that bricks a parent + "Café Assistant", # non-ASCII + "billing/refunds", # a slash + "deploy (staging)", # brackets + " padded ", # leading and trailing space + "emoji 🚀 agent", # astral plane + "tabs\tand\nnewlines", + "a" * 200, # long, but every character legal + ], +) +def test_every_authored_display_name_produces_a_valid_tool_name(display_name): + name = ReferenceToolConfig(slug="wf", name=display_name).tool_name + assert PROVIDER_TOOL_NAME.match(name), f"{display_name!r} -> {name!r}" + + +def test_a_clean_name_passes_through_unchanged(): + # The common case must not be disfigured: a name already matching the pattern is the name. + for clean in ["summarizer", "Support_Router", "billing-v2", "agent.v1", "A1"]: + assert ReferenceToolConfig(slug="wf", name=clean).tool_name == clean + + +def test_the_spaced_repro_becomes_the_obvious_thing(): + assert ( + ReferenceToolConfig(slug="wf", name="QA-v0.114.4 Helper").tool_name + == "QA-v0.114.4_Helper" + ) + assert ( + ReferenceToolConfig(slug="wf", name="Support Router").tool_name + == "Support_Router" + ) + + +def test_runs_of_disallowed_characters_collapse_to_one_separator(): + # "a___b" from "a b" would be noise; the model reads this name. + assert sanitize_tool_name("a b", fallback="wf") == "a_b" + assert sanitize_tool_name("a // b", fallback="wf") == "a_b" + + +def test_separators_are_trimmed_from_both_ends(): + assert sanitize_tool_name(" spaced ", fallback="wf") == "spaced" + assert sanitize_tool_name("...dots...", fallback="wf") == "dots" + assert sanitize_tool_name("---", fallback="wf") == "wf" + + +class TestFallback: + """A name that survives sanitization empty must still produce something callable.""" + + def test_a_symbol_only_name_falls_back_to_the_slug(self): + assert ReferenceToolConfig(slug="my-workflow", name="🚀🚀🚀").tool_name == ( + "my-workflow" + ) + assert ReferenceToolConfig(slug="my-workflow", name="///").tool_name == ( + "my-workflow" + ) + + def test_no_authored_name_uses_the_slug_as_before(self): + assert ReferenceToolConfig(slug="summarizer").tool_name == "summarizer" + + def test_a_slug_needing_sanitizing_is_sanitized_too(self): + assert sanitize_tool_name(None, fallback="my workflow") == "my_workflow" + + def test_a_last_resort_name_when_everything_sanitizes_empty(self): + # Not reachable through the model today (`slug` has min_length=1 and is a slug), but the + # helper must never return "" — an empty tool name is the same provider refusal. + assert sanitize_tool_name("///", fallback="***") == "tool" + assert PROVIDER_TOOL_NAME.match(sanitize_tool_name("", fallback="")) + + +class TestCollisions: + def test_two_names_that_sanitize_alike_stay_distinct(self): + pairs = [ + ("workflow.variant.a", "Support_Router"), + ("workflow.variant.b", "Support_Router"), + ] + resolved = disambiguate_tool_names(pairs) + assert resolved["workflow.variant.a"] != resolved["workflow.variant.b"] + for name in resolved.values(): + assert PROVIDER_TOOL_NAME.match(name) + assert name.startswith("Support_Router") + + def test_a_name_with_no_collision_is_left_alone(self): + # Only the colliding names are decorated, so the common case keeps the name the user + # recognizes from the UI. + pairs = [ + ("workflow.variant.a", "summarizer"), + ("workflow.variant.b", "router"), + ] + resolved = disambiguate_tool_names(pairs) + assert resolved == { + "workflow.variant.a": "summarizer", + "workflow.variant.b": "router", + } + + def test_the_discriminator_is_identity_derived_not_positional(self): + # An ordinal would renumber when the author reorders or deletes a sibling, changing a + # name the model may already have called earlier in the conversation. + forward = disambiguate_tool_names( + [("workflow.variant.a", "dup"), ("workflow.variant.b", "dup")] + ) + reversed_order = disambiguate_tool_names( + [("workflow.variant.b", "dup"), ("workflow.variant.a", "dup")] + ) + assert forward == reversed_order + + def test_a_three_way_collision_resolves_to_three_distinct_names(self): + pairs = [(f"workflow.variant.{c}", "dup") for c in "abc"] + resolved = disambiguate_tool_names(pairs) + assert len(set(resolved.values())) == 3 + + def test_an_unrelated_sibling_does_not_get_decorated_by_a_collision(self): + pairs = [ + ("workflow.variant.a", "dup"), + ("workflow.variant.b", "dup"), + ("workflow.variant.c", "unique"), + ] + resolved = disambiguate_tool_names(pairs) + assert resolved["workflow.variant.c"] == "unique" + + +def test_the_display_name_itself_is_never_rewritten(): + # Only the wire name changes. The UI, the config, and anything else reading `name` must still + # see exactly what the user typed. + config = ReferenceToolConfig(slug="wf", name="Support Router") + assert config.name == "Support Router" + assert config.tool_name == "Support_Router" + + +class TestResolverInteraction: + """Sanitizing must not make the resolver reject a configuration that is actually fine.""" + + def test_two_display_names_that_sanitize_alike_are_not_a_duplicate_error(self): + # The early declared-name pass runs BEFORE the adapter disambiguates, so it would see two + # `Support_Router` entries. Rejecting there would turn the fix into a different outage: + # the user could no longer save the pair at all. + from agenta.sdk.agents.tools.resolver import _validate_declared_config_names + + _validate_declared_config_names( + [ + ReferenceToolConfig(slug="a", name="Support Router"), + ReferenceToolConfig(slug="b", name="Support/Router"), + ] + ) + + def test_a_reference_tool_may_still_not_shadow_a_builtin(self): + # The other half of that pass is about a custom tool silently replacing a harness + # built-in, which sanitizing does nothing to excuse. + from agenta.sdk.agents.tools.errors import ReservedToolNameError + from agenta.sdk.agents.tools.resolver import _validate_declared_config_names + + with pytest.raises(ReservedToolNameError): + _validate_declared_config_names( + [ReferenceToolConfig(slug="wf", name="read")] + ) + + def test_a_genuine_duplicate_among_other_tool_kinds_still_raises(self): + from agenta.sdk.agents.tools import ClientToolConfig + from agenta.sdk.agents.tools.errors import DuplicateToolNameError + from agenta.sdk.agents.tools.resolver import _validate_declared_config_names + + with pytest.raises(DuplicateToolNameError): + _validate_declared_config_names( + [ + ClientToolConfig(name="dup", description="a"), + ClientToolConfig(name="dup", description="b"), + ] + ) + + +class TestDigestCollisions: + """A six-hex discriminator is 24 bits, so two identities can share it (CodeRabbit, #6412). + + When that happens on names that ALSO collide, the helper would return one name for both + entries and the final uniqueness check would reject a configuration this function documents as + unique — the same shadowing failure the discriminator exists to prevent, one layer down. + """ + + def test_the_digest_lengthens_until_the_group_is_distinct(self, monkeypatch): + from agenta.sdk.agents.tools import models + + # Force the 6-hex prefixes to collide while the full digests still differ, which is the + # real-world shape (a prefix collision, not a hash break). + forced = { + "workflow.variant.a": "aaaaaa" + "1" + "0" * 33, + "workflow.variant.b": "aaaaaa" + "2" + "0" * 33, + } + monkeypatch.setattr( + models, "_identity_digest", lambda identity: forced[identity] + ) + + resolved = models.disambiguate_tool_names( + [("workflow.variant.a", "dup"), ("workflow.variant.b", "dup")] + ) + assert len(set(resolved.values())) == 2, resolved + # It grew by exactly one character rather than jumping to the full digest. + assert resolved["workflow.variant.a"] == "dup_aaaaaa1" + assert resolved["workflow.variant.b"] == "dup_aaaaaa2" + + def test_the_width_is_a_property_of_the_set_not_the_order(self, monkeypatch): + from agenta.sdk.agents.tools import models + + forced = { + "workflow.variant.a": "aaaaaa" + "1" + "0" * 33, + "workflow.variant.b": "aaaaaa" + "2" + "0" * 33, + } + monkeypatch.setattr( + models, "_identity_digest", lambda identity: forced[identity] + ) + + forward = models.disambiguate_tool_names( + [("workflow.variant.a", "dup"), ("workflow.variant.b", "dup")] + ) + backward = models.disambiguate_tool_names( + [("workflow.variant.b", "dup"), ("workflow.variant.a", "dup")] + ) + assert forward == backward + + def test_a_deep_prefix_collision_still_resolves(self, monkeypatch): + from agenta.sdk.agents.tools import models + + # Identical for 20 characters: the width has to grow well past the default. + forced = { + "workflow.variant.a": "a" * 20 + "1" + "0" * 19, + "workflow.variant.b": "a" * 20 + "2" + "0" * 19, + } + monkeypatch.setattr( + models, "_identity_digest", lambda identity: forced[identity] + ) + + resolved = models.disambiguate_tool_names( + [("workflow.variant.a", "dup"), ("workflow.variant.b", "dup")] + ) + assert len(set(resolved.values())) == 2 + for name in resolved.values(): + assert PROVIDER_TOOL_NAME.match(name) + + def test_real_digests_need_no_extension(self): + # The common case must stay short and readable: distinct call_refs practically never + # share six hex characters, so the decorated name keeps its 6-char suffix. + resolved = disambiguate_tool_names( + [("workflow.variant.a", "dup"), ("workflow.variant.b", "dup")] + ) + for name in resolved.values(): + assert len(name) == len("dup_") + 6, name diff --git a/sdks/python/oss/tests/pytest/unit/test_batch_fold_stream_contract_routing.py b/sdks/python/oss/tests/pytest/unit/test_batch_fold_stream_contract_routing.py index 0738a96b0e..b5562e95a0 100644 --- a/sdks/python/oss/tests/pytest/unit/test_batch_fold_stream_contract_routing.py +++ b/sdks/python/oss/tests/pytest/unit/test_batch_fold_stream_contract_routing.py @@ -122,9 +122,7 @@ async def destroy(self) -> None: class _FakeBackend(Backend): - supported_harnesses = frozenset( - {HarnessKind.PI, HarnessKind.CLAUDE, HarnessKind.AGENTA} - ) + supported_harnesses = frozenset({HarnessKind.PI, HarnessKind.CLAUDE}) def __init__(self, *, events: List[Event]) -> None: self._events = events diff --git a/sdks/python/oss/tests/pytest/unit/test_invoke_real_handlers_negotiation_routing.py b/sdks/python/oss/tests/pytest/unit/test_invoke_real_handlers_negotiation_routing.py index 23e86c6d14..c177237ade 100644 --- a/sdks/python/oss/tests/pytest/unit/test_invoke_real_handlers_negotiation_routing.py +++ b/sdks/python/oss/tests/pytest/unit/test_invoke_real_handlers_negotiation_routing.py @@ -125,9 +125,7 @@ async def destroy(self) -> None: class _FakeBackend(Backend): - supported_harnesses = frozenset( - {HarnessKind.PI, HarnessKind.CLAUDE, HarnessKind.AGENTA} - ) + supported_harnesses = frozenset({HarnessKind.PI, HarnessKind.CLAUDE}) def __init__(self, *, events: List[Event], output: str = "") -> None: self._events = events diff --git a/sdks/python/pyproject.toml b/sdks/python/pyproject.toml index b2a6953177..4c2caf7ad9 100644 --- a/sdks/python/pyproject.toml +++ b/sdks/python/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "agenta" -version = "0.114.3" +version = "0.114.4" description = "Agenta is the open-source workspace for your agents. Build agents through chat, improve them with feedback, and share them with your team." readme = "README.md" requires-python = ">=3.11,<3.14" diff --git a/sdks/python/uv.lock b/sdks/python/uv.lock index 250eb8af89..703970e659 100644 --- a/sdks/python/uv.lock +++ b/sdks/python/uv.lock @@ -4,7 +4,7 @@ requires-python = ">=3.11, <3.14" [[package]] name = "agenta" -version = "0.114.3" +version = "0.114.4" source = { editable = "." } dependencies = [ { name = "agenta-client" }, @@ -85,7 +85,7 @@ dev = [ [[package]] name = "agenta-client" -version = "0.114.3" +version = "0.114.4" source = { editable = "../../clients/python" } dependencies = [ { name = "httpx" }, diff --git a/services/oss/tests/pytest/unit/agent/conftest.py b/services/oss/tests/pytest/unit/agent/conftest.py index f29455100c..18309dd136 100644 --- a/services/oss/tests/pytest/unit/agent/conftest.py +++ b/services/oss/tests/pytest/unit/agent/conftest.py @@ -90,7 +90,6 @@ def __init__( supported: Sequence[HarnessKind] = ( HarnessKind.PI, HarnessKind.CLAUDE, - HarnessKind.AGENTA, ), ) -> None: self.supported_harnesses = frozenset(supported) diff --git a/services/oss/tests/pytest/unit/agent/test_builtin_uri_binding.py b/services/oss/tests/pytest/unit/agent/test_builtin_uri_binding.py index 0d23a3aef9..d35024c548 100644 --- a/services/oss/tests/pytest/unit/agent/test_builtin_uri_binding.py +++ b/services/oss/tests/pytest/unit/agent/test_builtin_uri_binding.py @@ -73,7 +73,7 @@ def test_harness_capabilities_live_in_the_catalog_not_inspect_meta(): # for agent vs non-agent). They live in the `harnesses` catalog, keyed by harness, with # `capabilities` as a field. The frontend resolves them via `x-ag-harness-ref`. doc = harness_catalog_document() - assert set(doc) == {"pi_core", "pi_agenta", "claude", "codex"} + assert set(doc) == {"pi_core", "claude", "codex"} # Each record is {harness, capabilities: {...}}; claude reaches anthropic, Pi per-provider. assert doc["claude"]["harness"] == "claude" assert doc["claude"]["capabilities"]["models"]["anthropic"] diff --git a/services/oss/tests/pytest/unit/agent/test_invoke_handler.py b/services/oss/tests/pytest/unit/agent/test_invoke_handler.py index 0ffae99ada..bd9fafcbf7 100644 --- a/services/oss/tests/pytest/unit/agent/test_invoke_handler.py +++ b/services/oss/tests/pytest/unit/agent/test_invoke_handler.py @@ -286,14 +286,13 @@ async def test_invoke_cross_harness_same_body_divergent_configs( } bodies = [ await _invoke(harness, permission_default="deny", skills=[skill]) - for harness in ("pi_core", "pi_agenta", "claude") + for harness in ("pi_core", "claude") ] - pi_body, agenta_body, claude_body = bodies + pi_body, claude_body = bodies # (1) identical body regardless of harness assert ( pi_body - == agenta_body == claude_body == { "messages": [ @@ -305,11 +304,10 @@ async def test_invoke_cross_harness_same_body_divergent_configs( } ) - # (2) the three harness-shaped configs that reached the backend boundary, in call order - assert len(backend.created_configs) == 3 - pi_cfg, agenta_cfg, claude_cfg = backend.created_configs + # (2) the two harness-shaped configs that reached the backend boundary, in call order + assert len(backend.created_configs) == 2 + pi_cfg, claude_cfg = backend.created_configs pi_wire = pi_cfg.wire_tools() - agenta_wire = agenta_cfg.wire_tools() claude_wire = claude_cfg.wire_tools() # Pi carries its custom tool natively and always names every built-in on the deprecated @@ -324,19 +322,12 @@ async def test_invoke_cross_harness_same_body_divergent_configs( assert claude_wire["permissions"] == {"default": "deny"} assert "skills" not in claude_wire - # Agenta is Pi-with-an-opinion, and the opinion is prompt-shaped, not tool-shaped: the two - # share a tool wire. Skills are not tools, so they never appear in it either. - assert agenta_wire == pi_wire - assert "skills" not in agenta_wire - # skills ride the dedicated wire_skills seam, not the tool wire assert pi_cfg.wire_skills()["skills"][0]["name"] == "release-notes" - assert agenta_cfg.wire_skills()["skills"][0]["name"] == "release-notes" assert claude_cfg.wire_skills()["skills"][0]["name"] == "release-notes" # configs genuinely differ; the body's sameness is not a tautology assert pi_wire != claude_wire - assert agenta_cfg.wire_prompt() != pi_cfg.wire_prompt() async def test_stream_tool_resolution_failure_is_raised_before_backend_setup( diff --git a/services/oss/tests/pytest/unit/agent/test_template_shape_validation.py b/services/oss/tests/pytest/unit/agent/test_template_shape_validation.py index d9eb3961fe..9595927bff 100644 --- a/services/oss/tests/pytest/unit/agent/test_template_shape_validation.py +++ b/services/oss/tests/pytest/unit/agent/test_template_shape_validation.py @@ -14,7 +14,11 @@ import pytest -from agenta.sdk.agents import AgentTemplate, AgentTemplateShapeError +from agenta.sdk.agents import ( + AgentTemplate, + AgentTemplateShapeError, + InvalidHarnessKindError, +) from agenta.sdk.models.workflows import WorkflowServiceRequest from oss.src.agent import app @@ -180,3 +184,23 @@ async def test_handler_raises_on_flat_template(): messages=[{"role": "user", "content": "hi"}], parameters={"agent": {"harness": "claude", "sandbox": "daytona"}}, ) + + +@pytest.mark.parametrize("kind", [12345, "not_a_real_harness"]) +async def test_handler_raises_a_coded_400_on_an_unreadable_harness_kind(kind): + """F4: the shape is right, the harness is not. It must still be a named 400. + + This is the case a config persisted before the commit boundary existed still reaches. It + used to travel as far as ``make_harness`` and surface as an unhandled 500 whose body was + the enum's ``ValueError`` repr, so the caller learned neither the field nor the values it + could have sent. + """ + with pytest.raises(InvalidHarnessKindError) as caught: + await app._agent( + request=WorkflowServiceRequest(), + messages=[{"role": "user", "content": "hi"}], + parameters={"agent": {"harness": {"kind": kind}}}, + ) + + assert caught.value.code == 400 + assert "harness.kind" in caught.value.message diff --git a/services/pyproject.toml b/services/pyproject.toml index f726060078..254c64cf9f 100644 --- a/services/pyproject.toml +++ b/services/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "services" -version = "0.114.3" +version = "0.114.4" description = "Agenta Services (Chat & Completion)" requires-python = ">=3.11,<3.14" authors = [ diff --git a/services/runner/src/engines/sandbox_agent/applied-state.ts b/services/runner/src/engines/sandbox_agent/applied-state.ts index ed91127042..2f07bb2446 100644 --- a/services/runner/src/engines/sandbox_agent/applied-state.ts +++ b/services/runner/src/engines/sandbox_agent/applied-state.ts @@ -29,7 +29,7 @@ import { type FacetDigests, } from "../../lifecycle/desired-state.ts"; import type { AgentRunRequest } from "../../protocol.ts"; -import { configFingerprint } from "./session-identity.ts"; +import { configFieldDigests, configFingerprint } from "./session-identity.ts"; /** * The state an environment has successfully installed. Read-only to everyone except @@ -52,6 +52,12 @@ export interface AppliedEnvironmentState { * the same successful acquire, so it can never disagree with `configFingerprint`. */ readonly facets: FacetDigests; + /** + * Per-field digests of the same configuration (see `configFieldDigests`), so a config + * mismatch can log WHICH fields changed — names only, values never leave the hash. Stamped + * with the other two, so the three views describe one configuration. + */ + readonly fieldDigests: Record; } /** @@ -74,11 +80,17 @@ export class AppliedState implements AppliedStateOwner { #generation: number; #configFingerprint: string; #facets: FacetDigests; + #fieldDigests: Record; - constructor(configFingerprint: string, facets: FacetDigests) { + constructor( + configFingerprint: string, + facets: FacetDigests, + fieldDigests: Record, + ) { this.#generation = 1; this.#configFingerprint = configFingerprint; this.#facets = facets; + this.#fieldDigests = fieldDigests; } get appliedState(): AppliedEnvironmentState { @@ -87,6 +99,7 @@ export class AppliedState implements AppliedStateOwner { generation: this.#generation, configFingerprint: this.#configFingerprint, facets: { ...this.#facets }, + fieldDigests: { ...this.#fieldDigests }, }; } @@ -101,10 +114,12 @@ export class AppliedState implements AppliedStateOwner { commitApplied(result: { configFingerprint: string; facets: FacetDigests; + fieldDigests: Record; }): void { this.#generation += 1; this.#configFingerprint = result.configFingerprint; this.#facets = result.facets; + this.#fieldDigests = result.fieldDigests; } } @@ -117,9 +132,27 @@ export class AppliedState implements AppliedStateOwner { * real code can never reach. */ export function appliedStateForRequest(request: AgentRunRequest): AppliedState { - const fingerprint = configFingerprint(request); + const result = appliedResultForRequest(request); return new AppliedState( - fingerprint, - normalizeDesiredState(request, fingerprint).digests, + result.configFingerprint, + result.facets, + result.fieldDigests, ); } + +/** + * The three applied-state views of one request, for `commitApplied` callers. One place computes + * all of them, so no commit can stamp views of different configurations. + */ +export function appliedResultForRequest(request: AgentRunRequest): { + configFingerprint: string; + facets: FacetDigests; + fieldDigests: Record; +} { + const fingerprint = configFingerprint(request); + return { + configFingerprint: fingerprint, + facets: normalizeDesiredState(request, fingerprint).digests, + fieldDigests: configFieldDigests(request), + }; +} diff --git a/services/runner/src/engines/sandbox_agent/credential-preflight.ts b/services/runner/src/engines/sandbox_agent/credential-preflight.ts new file mode 100644 index 0000000000..f628985f05 --- /dev/null +++ b/services/runner/src/engines/sandbox_agent/credential-preflight.ts @@ -0,0 +1,226 @@ +/** + * Credential-substitution preflight for fresh Daytona sandboxes. + * + * THE FAULT THIS DETECTS (measured 2026-08-30, docs/design/daytona-secret-propagation/). + * On a Daytona run the model key never enters the sandbox: it is a Daytona Secret, and the + * sandbox holds a `dtn_secret_` placeholder Daytona substitutes into egress requests to + * the key's exact host. That wiring is BINARY PER SANDBOX: a healthy sandbox substitutes on + * its very first request (~2s after Secret creation), and a stuck sandbox never does — the + * raw placeholder reaches the provider for as long as anyone watches, a twin sandbox on the + * SAME Secret works immediately, and stop+start does not repair it. Measured 5 stuck of 20 + * fresh sandboxes (target eu); production showed ~3% over an earlier window, so the rate + * varies. Waiting therefore cannot help; only a fresh sandbox can. + * + * THE MECHANISM. Right after the sandbox is created, probe the credential's own endpoint + * from INSIDE the sandbox: POST `${baseUrl}/chat/completions` with the key env var as the + * bearer. Several consecutive raw-placeholder echoes convict the sandbox as STUCK; any other + * response means the header was substituted (a 400 for the junk body, a real 401 for a + * genuinely bad key) and the run may proceed. The preflight runs CONCURRENTLY with the rest + * of acquire (mounts, workspace, session open, ~10s), so a healthy sandbox pays nothing. + * + * ONLY A MASKED ECHO CONVICTS, AND THAT DISTINCTION IS THE WHOLE INSTRUMENT. A bare `dtn_` + * in the body proves nothing: Daytona's egress proxy also SCRUBS responses, rewriting real + * credential values back into `dtn_secret_` before they reach the sandbox. So an endpoint + * that echoes the Authorization header verbatim returns the full placeholder shape on a + * perfectly HEALTHY sandbox, and convicting on that would destroy both acquire attempts and + * fail a first turn whose real model call would have worked. What scrubbing cannot forge is a + * MASKED placeholder: a provider masks the key it received (LiteLLM's + * "Virtual Key expected. Received=dtn_****", OpenAI's "Incorrect API key provided: + * dtn_secr*****"), and a masked string no longer contains the real value for the scrubber to + * match — so a masked `dtn_` can only mean the raw placeholder really went out. This is the + * same correction that invalidated the first probe run; see the "CORRECTED" section of + * `docs/design/daytona-secret-propagation/README.md`. Every incident observed in production + * and in the 20-sandbox probe carried a masked echo, so the narrower signature costs no + * detection. + * + * WHAT A "STUCK" VERDICT DOES. The acquire path destroys the environment and retries ONCE + * with a brand-new sandbox, because the twin experiment proved a new sandbox on the same + * Secret works. The user sees a slower first turn instead of a failed one. + * + * THE GRACE IS 10 SECONDS, A DELIBERATE CHOICE BELOW DAYTONA'S ~30s BOUND. Their support + * (2026-08-31, confirming our report) said a sandbox may still start working within ~30s + * and must be recreated after that; "retrying or restarting the same one will not help." + * Our own 20 samples saw nothing land between 3s and 180s, every healthy sandbox answered + * on its FIRST probe, and their wiring fix is in progress on their side — so we convict at + * 10s and rebuild rather than hold every stuck user turn another 20s for a recovery nobody + * has observed. Decided by the product owner 2026-08-31; revisit only if a late recovery + * ever shows up in the preflight logs (it would log "substitution confirmed after N + * probes" with N > 1). + * + * SCOPE. Only a freshly created Daytona sandbox whose MODEL credential rides a Daytona + * Secret and whose connection declares an endpoint base URL (the custom OpenAI-compatible + * shape — every observed incident). A plaintext-env run has no placeholder; a reconnected + * sandbox already proved itself. + * + * AMBIGUITY FAILS OPEN. A probe that errors, returns nothing judgeable, or returns an + * UNMASKED placeholder-shaped echo returns "ok" and the run proceeds: the worst outcome is + * the pre-existing behavior, classified honestly by `classifyRunError`. Only consecutive, + * unambiguous masked raw-placeholder echoes convict. + */ + +/** + * Does this acquire deliver the run's MODEL credential as a Daytona Secret? + * + * The condition the preflight gates on, minus the endpoint — and the condition that arms the + * classifier's credential-race reading. Those two must not drift: the preflight can only SEE the + * race where the provider echoes what it received, but the race EXISTS wherever a model key rides + * a Secret on a fresh sandbox. Naming it once keeps that difference deliberate instead of + * accidental. + * + * A reconnect is excluded because the sandbox already proved itself, a local run because there is + * no Secret, and a plaintext-env run because there is no placeholder to substitute. + * + * Pure and unit-testable; `acquireEnvironment` itself cannot be driven without a live provider. + */ +export function deliversModelSecretOnCreate(input: { + isDaytona: boolean; + sandboxMode: string; + hasModelSecretCandidate: boolean; +}): boolean { + return ( + input.isDaytona && + input.sandboxMode === "create" && + input.hasModelSecretCandidate + ); +} + +export interface PreflightSandbox { + runProcess(request: { + command: string; + args?: string[]; + timeoutMs?: number; + }): Promise<{ exitCode?: number | null; stdout: string } | undefined>; +} + +/** The preflight's answer: proceed, or this sandbox will never substitute. */ +export type CredentialPreflightVerdict = "ok" | "stuck"; + +/** Thrown by the acquire path when the preflight convicts the sandbox. */ +export class SubstitutionStuckError extends Error { + constructor() { + super( + "This sandbox never received its credential-substitution wiring (raw placeholder " + + "echoed on every probe); a fresh sandbox is required.", + ); + this.name = "SubstitutionStuckError"; + } +} + +/** Total acquire attempts when a sandbox is convicted stuck: the original plus one retry. */ +export const STUCK_ACQUIRE_ATTEMPTS = 2; + +export interface CredentialPreflightInput { + sandbox: PreflightSandbox; + /** The custom connection's endpoint base URL (`modelConnection.endpoint.baseUrl`). */ + baseUrl: string; + /** The env var name holding the key in the sandbox (the Secret's placeholder). */ + apiKeyVar: string; + log: (message: string) => void; + /** Total budget from first probe to giving up (fail open). */ + budgetMs?: number; + /** Delay between probes. */ + pollMs?: number; + /** Injectable clock/sleep for tests. */ + now?: () => number; + sleep?: (ms: number) => Promise; +} + +/** See the module doc: 10s, deliberately below Daytona's ~30s keep-or-recreate bound. */ +const DEFAULT_BUDGET_MS = 10_000; +const DEFAULT_POLL_MS = 2_000; +const CURL_TIMEOUT_S = 8; + +/** + * The only echo that proves the raw placeholder went out: one the provider MASKED. + * + * See the module doc. Daytona's response scrubbing rewrites a real credential back into + * `dtn_secret_`, so an unmasked placeholder shape is equally consistent with a HEALTHY + * sandbox whose endpoint echoed the working key. Masking defeats the scrubber, because the + * masked string no longer contains the real value to match. + * + * First alternative: a `dtn_` token carrying a mask character, which covers both proven + * shapes (LiteLLM's `Received=dtn_****`, OpenAI's `dtn_secr*****`). Second: LiteLLM naming a + * `dtn_` key as what it received, which is proof on its own and survives a change of mask + * character; it mirrors `PLACEHOLDER_CREDENTIAL` in `errors.ts`. `*` is the only mask + * character trusted here — a truncation with `...` or `…` could equally be a cut-off scrubbed + * value, and ambiguity fails open. + */ +const MASKED_PLACEHOLDER_ECHO = + /dtn_[A-Za-z0-9_-]*\*|virtual key expected.*received=\s*dtn_/i; + +/** + * Resolve "ok" when the sandbox's model credential substitutes on the wire (or nothing can be + * judged — fail open), and "stuck" after enough consecutive raw-placeholder echoes. Never + * throws. + */ +export async function awaitCredentialSubstitution( + input: CredentialPreflightInput, +): Promise { + const now = input.now ?? Date.now; + const sleep = + input.sleep ?? + ((ms: number) => new Promise((resolve) => setTimeout(resolve, ms))); + const budgetMs = input.budgetMs ?? DEFAULT_BUDGET_MS; + const pollMs = input.pollMs ?? DEFAULT_POLL_MS; + const url = `${input.baseUrl.replace(/\/+$/, "")}/chat/completions`; + // The env var is expanded by the sandbox shell, so the placeholder value never appears in + // any runner-side string. `-d "{}"` makes an auth-first endpoint answer without a model call. + const script = + `curl -s -m ${CURL_TIMEOUT_S} -X POST ` + + `-H "Content-Type: application/json" ` + + `-H "Authorization: Bearer $${input.apiKeyVar}" ` + + `-d "{}" ${JSON.stringify(url)}`; + + const startedAt = now(); + for (let attempt = 1; ; attempt++) { + let body: string | undefined; + try { + const result = await input.sandbox.runProcess({ + command: "sh", + args: ["-c", script], + timeoutMs: (CURL_TIMEOUT_S + 4) * 1000, + }); + body = result?.stdout; + } catch (error) { + // The exec channel itself failed (sandbox tearing down, daemon hiccup): fail open. + input.log( + `[credential-preflight] probe errored, proceeding: ${String( + error instanceof Error ? error.message : error, + ).slice(0, 120)}`, + ); + return "ok"; + } + const elapsedMs = now() - startedAt; + if (!body || !MASKED_PLACEHOLDER_ECHO.test(body)) { + // Substituted, or the endpoint gave nothing this preflight can judge by — fail open + // either way. An unmasked placeholder shape lands here on purpose: scrubbing produces + // it from a healthy key too, so it is not evidence. Log it, because a stuck sandbox + // behind an echoing endpoint now passes the preflight and surfaces as the 401 instead. + if (body?.includes("dtn_")) { + input.log( + `[credential-preflight] unmasked placeholder-shaped echo (probe ${attempt}, ` + + `+${(elapsedMs / 1000).toFixed(1)}s): scrubbing produces this from a REAL key ` + + `too, so it convicts nothing; proceeding`, + ); + } else if (attempt > 1) { + input.log( + `[credential-preflight] substitution confirmed after ${attempt} probes ` + + `(${(elapsedMs / 1000).toFixed(1)}s)`, + ); + } + return "ok"; + } + if (elapsedMs + pollMs > budgetMs) { + input.log( + `[credential-preflight] STUCK: raw placeholder on all ${attempt} probes ` + + `(${(elapsedMs / 1000).toFixed(1)}s); this sandbox will never substitute`, + ); + return "stuck"; + } + input.log( + `[credential-preflight] raw placeholder echoed (probe ${attempt}, ` + + `+${(elapsedMs / 1000).toFixed(1)}s)`, + ); + await sleep(pollMs); + } +} diff --git a/services/runner/src/engines/sandbox_agent/daytona-secret-provider.ts b/services/runner/src/engines/sandbox_agent/daytona-secret-provider.ts index f156d0d437..0e6336cec0 100644 --- a/services/runner/src/engines/sandbox_agent/daytona-secret-provider.ts +++ b/services/runner/src/engines/sandbox_agent/daytona-secret-provider.ts @@ -197,7 +197,7 @@ export function daytonaWithProcessLocalSecrets( // A Secret remains mounted until Daytona confirms the sandbox is absent. Never reverse this // order, including timer cleanup and create compensation after an id was returned. await destroySandboxIdempotently(activeProvider, sandboxId); - await deleteDaytonaSecrets(entry.allocation, api); + await deleteDaytonaSecrets(entry.allocation, api, log); if (registry.get(sandboxId) === entry) registry.delete(sandboxId); if (currentAllocation === entry.allocation) { currentAllocation = undefined; @@ -208,14 +208,19 @@ export function daytonaWithProcessLocalSecrets( const facade: ProcessLocalDaytonaSecretProvider = { name: "daytona", async create(...args: unknown[]): Promise { - const allocation = await allocateDaytonaSecrets(plan, api); + const allocation = await allocateDaytonaSecrets( + plan, + api, + undefined, + log, + ); try { provider = buildProvider(allocation.attachments); } catch (cause) { // buildProvider is synchronous and failed before any remote create call, so absence is // proven and compensation may safely remove the newly allocated Secrets. try { - await deleteDaytonaSecrets(allocation, api); + await deleteDaytonaSecrets(allocation, api, log); } catch (cleanupError) { throw new AggregateError( [cause, cleanupError], diff --git a/services/runner/src/engines/sandbox_agent/daytona-secrets.ts b/services/runner/src/engines/sandbox_agent/daytona-secrets.ts index f2fa800b90..e56cd458fd 100644 --- a/services/runner/src/engines/sandbox_agent/daytona-secrets.ts +++ b/services/runner/src/engines/sandbox_agent/daytona-secrets.ts @@ -143,12 +143,24 @@ function generatedName(candidate: DaytonaSecretCandidate): string { return `agenta_${randomBytes(18).toString("hex")}_${candidate.ordinal}`; } -/** Allocate every Secret before sandbox create, compensating in reverse order on any failure. */ +/** + * Allocate every Secret before sandbox create, compensating in reverse order on any failure. + * + * `log` gets one line per allocation and deletion with the COUNT, the allowed HOSTS, and the + * elapsed time — never an id, a generated name, a placeholder, or a value. This is a deliberate, + * narrow exception to the delivery layer's log-nothing rule: Daytona applies a new Secret's + * substitution rule asynchronously, and diagnosing a raw-placeholder 401 (see + * `classifyRunError`'s `credential_delivery_failed`) needs the create/delete timeline that today + * has to be reconstructed by inference from eviction lines. Hosts are config, not credential + * material (the same hosts appear in the vault UI and in the resolved-model log line). + */ export async function allocateDaytonaSecrets( plan: DaytonaSecretPlan, api: DaytonaSecretApi, nameFor: (candidate: DaytonaSecretCandidate) => string = generatedName, + log: (message: string) => void = () => {}, ): Promise { + const startedAt = Date.now(); const created: DaytonaSecretRecord[] = []; const attachments: Record = {}; const mcpHeaderPlaceholders: Record> = {}; @@ -186,6 +198,13 @@ export async function allocateDaytonaSecrets( ] = secret.placeholder; } } + if (created.length > 0) { + const hosts = [...new Set(plan.candidates.map((c) => c.allowedHost))]; + log( + `[daytona-secrets] allocated n=${created.length} hosts=[${hosts.join(",")}] ` + + `ms=${Date.now() - startedAt}`, + ); + } return { attachments, mcpHeaderPlaceholders, created, bySlot }; } catch (cause) { const cleanupFailures: unknown[] = []; @@ -210,7 +229,9 @@ export async function allocateDaytonaSecrets( export async function deleteDaytonaSecrets( allocation: DaytonaSecretAllocation, api: DaytonaSecretApi, + log: (message: string) => void = () => {}, ): Promise { + const startedAt = Date.now(); const failures: unknown[] = []; for (const secret of [...allocation.created].reverse()) { try { @@ -225,4 +246,13 @@ export async function deleteDaytonaSecrets( "Daytona Secret cleanup was incomplete.", ); } + if (allocation.created.length > 0) { + const hosts = [ + ...new Set(allocation.created.flatMap((s) => s.hosts ?? [])), + ]; + log( + `[daytona-secrets] deleted n=${allocation.created.length} hosts=[${hosts.join(",")}] ` + + `ms=${Date.now() - startedAt}`, + ); + } } diff --git a/services/runner/src/engines/sandbox_agent/environment.ts b/services/runner/src/engines/sandbox_agent/environment.ts index 4f4323b924..fa92543e92 100644 --- a/services/runner/src/engines/sandbox_agent/environment.ts +++ b/services/runner/src/engines/sandbox_agent/environment.ts @@ -62,6 +62,12 @@ import { } from "./daytona.ts"; import { applyCodexMode, resolveCodexMode } from "./codex-mode.ts"; import { conciseError } from "./errors.ts"; +import { + awaitCredentialSubstitution, + deliversModelSecretOnCreate, + STUCK_ACQUIRE_ATTEMPTS, + SubstitutionStuckError, +} from "./credential-preflight.ts"; import { PI_MODEL_PROVIDER_OVERRIDE_ENV } from "../../extensions/model-provider-override.ts"; import { daytonaCredentialDeliveryPort, @@ -299,6 +305,39 @@ export async function acquireEnvironment( signal?: AbortSignal, presignedMount?: MountCredentials | null, emit?: EmitEvent, +): Promise { + // A sandbox the preflight convicts as stuck (no Secret substitution wiring, a permanent + // per-sandbox fault) is already destroyed by the failure path; a FRESH sandbox on the same + // Secret works, so one retry converts a would-be failed first turn into a slower one. + for (let attempt = 1; ; attempt++) { + const result = await acquireEnvironmentOnce( + request, + deps, + signal, + presignedMount, + emit, + ); + if ( + result.ok || + !result.stuckSubstitution || + attempt >= STUCK_ACQUIRE_ATTEMPTS || + signal?.aborted + ) { + return result; + } + process.stderr.write( + `[sandbox-agent] stuck-substitution sandbox destroyed; rebuilding fresh ` + + `(attempt ${attempt + 1}/${STUCK_ACQUIRE_ATTEMPTS})\n`, + ); + } +} + +async function acquireEnvironmentOnce( + request: AgentRunRequest, + deps: SandboxAgentDeps = {}, + signal?: AbortSignal, + presignedMount?: MountCredentials | null, + emit?: EmitEvent, ): Promise { emit?.({ type: "data", @@ -584,6 +623,41 @@ export async function acquireEnvironment( // a process KILL; removed in `destroy` on every normal exit so it is never double-deleted. if (environment.sandbox) inFlightSandboxes.add(environment); + // CREDENTIAL PREFLIGHT (fresh Daytona sandboxes with an opaque model key and a declared + // endpoint). Kicked off HERE, right after the sandbox exists, and awaited at the very end of + // acquire, so it runs concurrently with the mounts/workspace/session work below and the + // common case pays nothing. See `credential-preflight.ts` for the race it closes. + const modelSecretCandidate = + plan.credentials.daytonaSecretPlan?.candidates.find( + (candidate) => candidate.consumer.kind === "model", + ); + const preflightBaseUrl = request.modelConnection?.endpoint?.baseUrl?.trim(); + // Record the delivery moment for the 401 classifier. Same condition as the preflight below, + // minus the endpoint: the race exists wherever a model key rides a Secret on a fresh sandbox, + // but the preflight can only SEE it where the provider echoes what it received. A direct + // Anthropic endpoint echoes nothing, so on that path the classifier is the only guard. + if ( + deliversModelSecretOnCreate({ + isDaytona: plan.isDaytona, + sandboxMode: acquiredSandbox.mode, + hasModelSecretCandidate: Boolean(modelSecretCandidate), + }) + ) { + environment.modelSecretDeliveredAt = Date.now(); + } + const credentialPreflight = + plan.isDaytona && + acquiredSandbox.mode === "create" && + modelSecretCandidate && + preflightBaseUrl + ? (deps.awaitCredentialSubstitution ?? awaitCredentialSubstitution)({ + sandbox: environment.sandbox, + baseUrl: preflightBaseUrl, + apiKeyVar: modelSecretCandidate.binding.name, + log: logger, + }) + : undefined; + // On Daytona, push the harness login, the extension, and AGENTS.md into the remote sandbox. // For a non-Pi harness with executable tools, also push the in-sandbox stdio MCP shim // assets (bundle + public-specs file): a non-Pi harness in the sandbox cannot reach the @@ -1113,6 +1187,15 @@ export async function acquireEnvironment( routePermissionRequestToActiveTurn(environment, req), ); + if (credentialPreflight) { + const preflightAwaitStartedAt = Date.now(); + const verdict = await credentialPreflight; + timingLog("credential_preflight", preflightAwaitStartedAt); + // Throwing takes the shared teardown below (sandbox destroyed, Secrets deleted), and the + // catch marks the result so the acquire wrapper retries once with a fresh sandbox. + if (verdict === "stuck") throw new SubstitutionStuckError(); + } + timingLog("acquire_total", acquireStartedAt); emit?.({ type: "data", @@ -1122,6 +1205,13 @@ export async function acquireEnvironment( }); return { ok: true, env: environment }; } catch (err) { + // DELIBERATELY WITHOUT `daytonaCredentialFresh`, unlike the two call sites in `run-turn.ts`. + // Acquire INSTALLS the model credential but never exercises it: the first model call belongs + // to the turn. The one credential-shaped failure this path can raise is the preflight's + // `SubstitutionStuckError`, which already has its own honest answer below (rebuild once). + // Wiring the predicate here would also need the once-per-session counter, which lives in the + // turn path — without it a genuinely bad key could loop. If a model-touching step is ever + // added to acquire, this site needs BOTH the predicate and that counter. const error = conciseError( err, plan.harness, @@ -1131,6 +1221,9 @@ export async function acquireEnvironment( // Mirror today's shared teardown: no otel exists yet during acquire, so there is no partial // trace to flush — just run the incrementally-registered finalizers and surface the error. await environment.destroy({ reason: "failed-turn" }); + if (err instanceof SubstitutionStuckError) { + return { ok: false, error, stuckSubstitution: true }; + } return { ok: false, error }; } } diff --git a/services/runner/src/engines/sandbox_agent/errors.ts b/services/runner/src/engines/sandbox_agent/errors.ts index 8fde515632..bee78aeb63 100644 --- a/services/runner/src/engines/sandbox_agent/errors.ts +++ b/services/runner/src/engines/sandbox_agent/errors.ts @@ -21,8 +21,24 @@ const PROVIDER_KEY_LABELS: Record = { * resolved connection). When it is absent, fall back to the harness default * — Claude is always Anthropic; every other harness defaults to OpenAI, matching the old * behavior for that path only. + * + * A CUSTOM deployment overrides the family label entirely: its provider family is "openai" + * because the endpoint speaks the OpenAI dialect, not because the key is an OpenAI key — a + * Gemini run through an OpenAI-compatible proxy must not read "add the project's OpenAI key". + * The hint names the connection instead, which is where that key actually lives. */ -function keyHintFor(provider: string | undefined, harness: string): string { +function keyHintFor( + provider: string | undefined, + harness: string, + connection?: ConciseErrorOptions["connection"], +): string { + if (connection?.deployment === "custom") { + // NEUTRAL on purpose: the runner cannot tell a user-created connection from a managed one + // (the seeded starter-credits connection is write-only and hidden from Settings), so naming + // the slug can both leak an internal identifier and instruct the user to edit a connection + // they cannot see. Review finding on #6362. + return "the model connection's API key"; + } const label = provider ? PROVIDER_KEY_LABELS[provider.toLowerCase()] : undefined; @@ -44,6 +60,7 @@ export type RunErrorCode = | "starter_credits_exhausted" | "starter_credits_program_paused" | "starter_credits_unavailable" + | "credential_delivery_failed" | "rate_limited"; /** One failed run, condensed: the line the user reads plus the class a client can act on. */ @@ -53,13 +70,11 @@ export interface ClassifiedRunError { } /* - * TODO(copy: owner) — the five strings below are PLACEHOLDERS. They are the first product copy the - * runner puts in front of an end user (every other line here is an operator hint), so the final - * wording is the product owner's to write. Keep them short, plain, and free of provider/proxy - * mechanics: the user cannot act on which service refused, only on what to do next. - * - * They are deliberately NOT prefixed with the harness name the way the operator hints are — the - * reader of these is the person chatting, to whom "claude:" is noise. + * Product copy, settled 2026-08-31 for v0.114.4. These strings are the first product copy the + * runner shows to an end user; every other line here is an operator hint. Keep them short and + * plain, with no provider or proxy mechanics: the user cannot act on which service refused, + * only on what to do next. They carry no harness-name prefix, unlike the operator hints, + * because the reader is the person in the chat. */ const STARTER_CREDITS_EXHAUSTED_MESSAGE = "Your free Agenta credits are used up. Add your own provider key to keep going."; @@ -71,6 +86,8 @@ const PROVIDER_RATE_LIMITED_MESSAGE = "Too many requests to the model provider right now. Try again in a moment."; const STARTER_CREDITS_UNAVAILABLE_MESSAGE = "Agenta credits are temporarily unavailable. Try again in a moment."; +const CREDENTIAL_DELIVERY_FAILED_MESSAGE = + "A temporary issue kept this run's credentials from reaching the model. Send the message again."; /* * Recognition is matched on the BODY, never on the HTTP status alone: 429 covers admission-time @@ -96,6 +113,157 @@ const PROXY_RATE_LIMIT = /** The upstream provider's own quota refusal (Vertex/Google shape), distinct from a billing stop. */ const PROVIDER_QUOTA_EXHAUSTED = /resource_exhausted|quota exceeded/i; +/** + * The provider received the sandbox's opaque credential PLACEHOLDER instead of the real key. + * + * On a Daytona run the real model key never enters the sandbox: it is stored as a Daytona Secret + * and the sandbox holds a `dtn_secret_` placeholder that Daytona substitutes into egress + * requests to the key's exact host. That substitution propagates asynchronously with no + * confirmation signal, and when a sandbox's FIRST outbound call beats it (observed live at 10-24s + * after Secret creation), the raw placeholder reaches the provider and is refused with a 401. + * The user's key is fine, so the add-a-key advice would be wrong three ways; this is its own + * transient class. The first alternative matches LiteLLM's refusal of a non-`sk-` bearer + * ("LiteLLM Virtual Key expected. Received=dtn_****…"); the second matches any provider that + * echoes the placeholder itself. + * + * Both alternatives are SELF-EVIDENCING: each names the placeholder in a shape only the delivery + * layer produces, so neither needs corroboration. They stay anchored on the literal `dtn_` + * namespace, Daytona's placeholder prefix, which cannot appear in an `sk-` provider key. + */ +const PLACEHOLDER_CREDENTIAL = + /virtual key expected.*received=dtn_|dtn_secret_/i; + +/** + * A provider echoing the placeholder MASKED, which the signature above cannot see. + * + * OpenAI answers a direct call with "Incorrect API key provided: dtn_secr***************cdef" — + * the mask truncates before the literal `dtn_secret_`, so without this every direct OpenAI + * placeholder 401 was blamed on the user's key. It mirrors `MASKED_PLACEHOLDER_ECHO` in + * `credential-preflight.ts`, and `*` is the only mask character trusted here for the same reason + * there: a `...`/`…` truncation could equally be a cut-off scrubbed value. + * + * WHY IT IS SHAPED THIS TIGHTLY, AND WHY IT NEEDS CORROBORATION. Unlike the two above, this + * pattern is a guess about formatting rather than a quoted protocol string, so it is the one that + * can be spoofed by ordinary text. The stem is `{4,}` and the mask `{3,}` so a literal glob like + * `dtn_*` — a perfectly normal thing to find in a path, a filter, or a log line — cannot match; + * a real mask is many characters wide. And the caller requires AUTH_REFUSAL alongside it, so a + * hypothetical customer key spelled `dtn_customer_***` inside an unrelated error is not read as a + * delivery fault. Corroboration costs nothing here: an unsubstituted placeholder is only ever + * observed as a credential refusal. + */ +const MASKED_PLACEHOLDER_ECHO = /dtn_[A-Za-z0-9_-]{4,}\*{3,}/i; + +/** + * A refusal of the credential itself, whatever the provider calls it. + * + * `401` must stand alone (not digit-adjacent) so it doesn't false-match a bare HTTP status code + * embedded in an unrelated number — e.g. a `Date.now()`-based path/id that happens to contain + * "401" as a substring (a real, timestamp-dependent flake this caused). + */ +const AUTH_REFUSAL = + /authentication required|invalid api key|unauthorized|(? failed: HTTP 401`), attachment fetch, attachment claim, session-records query, + * and session-records persist. Without this exclusion, any one of them landing inside the + * propagation window would consume the session's single credential-race report and print retry + * guidance for a failure a retry cannot fix — and the genuine race that followed would then get + * the add-a-key copy, which is the original bug wearing a disguise. + * + * WHY THIS IS SOUND RATHER THAN A GUESS. Each of the five is greppable in `services/runner/src` + * and prefixed AT ITS THROW SITE precisely so it can be recognized here — four of them threw a + * bare `HTTP ` until this change and were genuinely indistinguishable from a provider + * refusal. The alternative, plumbing a typed provider-response provenance signal through the + * harness boundary into `ConciseErrorOptions`, is the right long-term shape and a large change; + * naming the emitters costs one regex and one word per throw site. + * + * WHAT IS DELIBERATELY NOT HERE. Mount, geesefs and otel failures are NOT in this set, for two + * independent reasons. They never arrive as classifier input: those sites build their message + * AROUND `conciseError(err, ...)`, so the prefix is added after classification and the classifier + * only ever sees the inner error. And matching them is actively unsafe — none is a prefixed + * emitter, so the patterns would have to be loose, and a loose `mount failed` matches inside + * "the requested amount failed to authorize" or "paramount failed" while a bare `otel` matches + * inside "hotel-search". Excluding a provider-shaped string is the WORSE direction of this bug: + * it hands a genuine race the add-a-key copy, which is the failure this whole class exists to + * prevent. If one ever does prove reachable, re-add it anchored with `\b`. + * + * THE STANDING OBLIGATION: a new authenticated call inside the turn must prefix its failure, or it + * silently rejoins this hazard. That is why the five throw sites carry a comment pointing back. + */ +const RUNNER_INTERNAL_401 = + /tool call .*failed: HTTP|attachment (?:fetch|claim) failed|session records (?:query|persist) failed/i; + +/** + * How long after a Daytona Secret is delivered a credential refusal is still better explained by + * propagation than by the key. + * + * Daytona's support puts the outer bound at ~30s, our own samples saw healthy substitution in + * ~2s, and the preflight convicts a stuck sandbox at 10s. The first model call lands after + * acquire, so the window has to outlast acquire itself; 60s covers that with margin while + * staying far short of a warm sandbox's later turns, where a 401 really is about the key. + * + * ACCEPTED LIMITATION: an unusually slow acquire pushes a GENUINE race past 60s and it gets the + * add-a-key advice instead. That is the right way round to be wrong. Substitution propagates in + * 10-24s, so a refusal arriving a full minute after delivery is far more likely a real bad key — + * exactly the reader the fallback advice serves. The cost when it does misfire is one turn shown + * the pre-fix copy, on a run whose retry lands on a fresh sandbox anyway. + */ +const CREDENTIAL_PROPAGATION_WINDOW_MS = 60_000; + +/** + * How many times one conversation may be told its credentials did not reach the model. + * + * A credential race and a genuinely wrong key look identical on a direct provider: both are a 401 + * with no placeholder echo. The retry copy is the right answer for the race — the failed turn + * DELETES the sandbox, so the retry lands on a fresh one and the per-sandbox fault is gone — but + * it is a trap for a bad key, which would be told to retry forever. One report per session bounds + * that: the second identical failure, on a second fresh sandbox, is far better explained by the + * key, so it falls through to the ordinary add-a-key advice. A bad key costs exactly one wasted + * retry; a real race still recovers silently. + */ +export const CREDENTIAL_RACE_REPORTS_PER_SESSION = 1; + +/** + * Whether a credential refusal falls inside the propagation window of a Daytona-delivered key. + * + * Exported for the call sites that build the predicate, and so a test can pin the window. + */ +export function withinCredentialPropagationWindow( + deliveredAt: number | undefined, + now: number = Date.now(), +): boolean { + return ( + deliveredAt !== undefined && + now - deliveredAt < CREDENTIAL_PROPAGATION_WINDOW_MS + ); +} + /** The proxy answered but cannot reach its own store, or was not reachable at all. */ const PROXY_NO_DATABASE = /no_db_connection/i; const CONNECTION_FAILURE = @@ -117,6 +285,25 @@ export interface ConciseErrorOptions { * Lazy so the check (a stat) only runs on the error path it explains. */ authFault?: () => string | undefined; + /** + * The run's named connection (wire `connection.slug`) and resolved deployment + * (`modelConnection.deployment`), when the caller knows them. A custom deployment carries the + * provider family "openai" for its DIALECT, so without this the auth hint names a key the + * user never configured; with it, the hint names the connection the key lives on. + */ + connection?: { slug?: string; deployment?: string }; + /** + * Whether this run's MODEL credential rode a Daytona Secret delivered recently enough that + * substitution may not have propagated. Lazy, like `authFault`: only the refusal path asks. + * + * This is the ONLY signal available on a direct provider. The body-echo signature above sees + * the race only when the provider names the placeholder it received, which the credits proxy + * does ("Received=dtn_****") and a direct endpoint does not — api.anthropic.com answers a + * bad bearer with "Invalid bearer token" and no echo at all, and a masked OpenAI echo + * ("dtn_secr*****") no longer contains the literal `dtn_secret_` the signature looks for. + * Without this option every direct-path placeholder 401 is blamed on the user's key. + */ + daytonaCredentialFresh?: () => boolean; } /** @@ -135,7 +322,7 @@ export function classifyRunError( ): ClassifiedRunError { const raw = err instanceof Error ? err.message : String(err); const msg = raw.split("\n")[0].trim(); - const keyHint = keyHintFor(provider, harness); + const keyHint = keyHintFor(provider, harness, options.connection); // A budget refusal is checked first: it is the most specific reading of a 429, and its body also // trips the rate-limit and quota matchers below. if (BUDGET_REFUSAL.test(raw)) { @@ -178,13 +365,52 @@ export function classifyRunError( if (PROXY_RATE_LIMIT.test(raw)) { return { message: RATE_LIMITED_MESSAGE, code: "rate_limited" }; } - // `401` must stand alone (not digit-adjacent) so it doesn't false-match a bare HTTP status - // code embedded in an unrelated number — e.g. a `Date.now()`-based path/id that happens to - // contain "401" as a substring (a real, timestamp-dependent flake this caused). + // Before the generic auth branch: a placeholder-shaped refusal IS a 401, but its cause is + // credential delivery, not the user's key, and the add-a-key advice would be false. + // + // `PLACEHOLDER_CREDENTIAL` is self-evidencing and stands alone: it quotes a protocol string only + // the delivery layer produces. `MASKED_PLACEHOLDER_ECHO` is NOT — it is a guess about formatting, + // so it is corroborated by `AUTH_REFUSAL` and only the pair of them together is evidence. + // + // DELIBERATELY NOT SUBJECT TO THE PER-SESSION REPORT BUDGET, unlike the branch below. That + // budget exists because a bare 401 cannot distinguish a delivery race from a genuinely wrong + // key, so the honest-retry reading has to be spent sparingly. A body that ECHOES the placeholder + // carries its own proof: a real user key never contains `dtn_`, so every such refusal IS a + // delivery failure, however many times it happens. Capping it would eventually tell a user with + // a perfectly good key to go add one — the exact wrong answer this class exists to prevent. + if ( + PLACEHOLDER_CREDENTIAL.test(raw) || + (MASKED_PLACEHOLDER_ECHO.test(raw) && AUTH_REFUSAL.test(raw)) + ) { + return { + message: CREDENTIAL_DELIVERY_FAILED_MESSAGE, + code: "credential_delivery_failed", + }; + } + // Still before the generic auth branch, and the direct-provider half of the case above: the + // refusal carries no placeholder because the provider does not echo what it received, so the + // only evidence is that this run's key WAS a Daytona Secret delivered moments ago. Same class, + // same honest copy — the alternative is telling a user with a valid key to add one. + // + // `PROVIDER_401`, not `AUTH_REFUSAL`: this branch spends the session's one report and prints + // retry guidance, so it must not fire for an authorization failure that merely says + // "unauthorized" somewhere unrelated. See the note on `PROVIDER_401`. + // + // And not a 401 the RUNNER itself produced. A tool call, an attachment fetch, or a + // session-records query can answer 401 inside the same window and reach this same catch; the + // status in the string is the runner's own prefix, not the provider's response, so the only way + // to tell them apart is to name them. See `RUNNER_INTERNAL_401`. if ( - /authentication required|invalid api key|unauthorized/i.test(raw) || - /(? { + if (!guidance || request.gatewayGuidance?.carrier !== carrier) + return authored; + return authored ? `${guidance}\n\n${authored}` : guidance; + }; const appendSystemPrompt = isPi - ? request.appendSystemPrompt?.trim() || undefined + ? spliceGuidance( + "appendSystemPrompt", + request.appendSystemPrompt?.trim() || undefined, + ) : undefined; // Debug assertions: the derived run state must be self-consistent before the engine acts on @@ -759,7 +787,10 @@ export function buildRunPlan( prompt: { text: prompt, turnText: buildTurnText(request, log), - agentsMd: request.agentsMd?.trim() || undefined, + agentsMd: spliceGuidance( + "agentsMd", + request.agentsMd?.trim() || undefined, + ), systemPrompt, appendSystemPrompt, hasSystemPrompt: !!(systemPrompt || appendSystemPrompt), diff --git a/services/runner/src/engines/sandbox_agent/run-turn.ts b/services/runner/src/engines/sandbox_agent/run-turn.ts index 9aabdcfbb8..b72265165a 100644 --- a/services/runner/src/engines/sandbox_agent/run-turn.ts +++ b/services/runner/src/engines/sandbox_agent/run-turn.ts @@ -62,7 +62,11 @@ import { type AcpPromptBlock, } from "./attachments.ts"; import { describeCodexSubscriptionAuthFault } from "./codex-assets.ts"; -import { classifyRunError } from "./errors.ts"; +import { + classifyRunError, + CREDENTIAL_RACE_REPORTS_PER_SESSION, + withinCredentialPropagationWindow, +} from "./errors.ts"; import { PAUSED, PendingApprovalPauseController } from "./pause.ts"; import { capturePiTranscriptCursor, @@ -144,6 +148,47 @@ export async function runTurn( // (honest interrupted transcript, keep-warm) instead of falling through to the error catch. const CANCELLED = Symbol("cancelled"); const continuityStore = deps.sessionContinuityStore ?? sessionContinuityStore; + /** + * Should a credential refusal this turn be reported as a delivery race rather than a bad key? + * + * Two conditions, and it MUTATES the session's report counter, so it is called only from the + * classifier's credential-refusal branch — never speculatively. The window says the race is + * physically possible (a Daytona Secret delivered to this sandbox moments ago); the counter + * says the explanation has not been spent on this conversation already. + * + * A run with no session id cannot be counted, so it always gets the honest copy. There is no + * conversation to loop within, and the alternative — blaming a key that may be perfectly good — + * is the failure this whole branch exists to remove. + */ + const reportCredentialRace = (): boolean => { + if (!withinCredentialPropagationWindow(env.modelSecretDeliveredAt)) { + return false; + } + if (!sessionId) { + logger( + "[credential-race] credential_delivery_failed (no session id, uncounted): a Daytona " + + "Secret for this run was delivered inside the propagation window", + ); + return true; + } + const occurrence = continuityStore.noteCredentialRaceReport(sessionId); + if (occurrence > CREDENTIAL_RACE_REPORTS_PER_SESSION) { + logger( + `[credential-race] NOT credential_delivery_failed (occurrence ${occurrence} > ` + + `${CREDENTIAL_RACE_REPORTS_PER_SESSION} for this session): a second fresh sandbox ` + + "refused the same way, so the key is the better explanation; falling through to the " + + "model-authentication advice", + ); + return false; + } + logger( + `[credential-race] credential_delivery_failed (occurrence ${occurrence}/` + + `${CREDENTIAL_RACE_REPORTS_PER_SESSION} for this session): the model credential was ` + + "delivered as a Daytona Secret inside the propagation window, so this refusal is " + + "delivery, not the user's key", + ); + return true; + }; const turnStartedAt = new Date().toISOString(); // `turn_index` is a true conversation-turn counter, not an acquire counter: it advances once per completed turn across every environment serving the session. // The shared store advances only on `record()` (paused turns record nothing), so park-and-resume consumes one index; compute it at turn start because a warm environment serves many turns. @@ -1292,6 +1337,17 @@ export async function runTurn( new Error(swallowedPiError), plan.harness, request.modelConnection?.provider, + { + connection: { + slug: request.connection?.slug, + deployment: request.modelConnection?.deployment, + }, + // The recovery path needs the same signal as the catch below. Pi records the + // provider's refusal in its transcript and ends the turn cleanly, so a credential + // race that arrives THIS way is the identical failure wearing a different shape — + // and without the predicate it would still be reported as the user's key problem. + daytonaCredentialFresh: reportCredentialRace, + }, ); swallowedError = classified.message; run.recordError(swallowedError, request.modelConnection?.provider); @@ -1373,7 +1429,14 @@ export async function runTurn( err, plan.harness, request.modelConnection?.provider, - { authFault: () => describeCodexSubscriptionAuthFault(plan) }, + { + authFault: () => describeCodexSubscriptionAuthFault(plan), + connection: { + slug: request.connection?.slug, + deployment: request.modelConnection?.deployment, + }, + daytonaCredentialFresh: reportCredentialRace, + }, ); const error = classified.message; await harnessTrace.cancelBeforeDrain(); diff --git a/services/runner/src/engines/sandbox_agent/runtime-contracts.ts b/services/runner/src/engines/sandbox_agent/runtime-contracts.ts index e1f34902b4..47c107f672 100644 --- a/services/runner/src/engines/sandbox_agent/runtime-contracts.ts +++ b/services/runner/src/engines/sandbox_agent/runtime-contracts.ts @@ -19,6 +19,7 @@ import { createAcpFetch } from "./acp-fetch.ts"; import { type ParkedApprovalGateType } from "./acp-interactions.ts"; import { signAgentMountCredentials } from "./agent-mount.ts"; import { probeCapabilities } from "./capabilities.ts"; +import { awaitCredentialSubstitution } from "./credential-preflight.ts"; import { createToolCallCorrelationIndex } from "./client-tools.ts"; import { buildDaemonEnv, resolveDaemonBinary } from "./daemon.ts"; import { createCookieFetch, prepareDaytonaPiAssets } from "./daytona.ts"; @@ -65,6 +66,7 @@ export interface SandboxAgentDeps extends BuildRunPlanDeps { prepareDaytonaPiAssets?: typeof prepareDaytonaPiAssets; uploadToolMcpAssets?: typeof uploadToolMcpAssets; probeCapabilities?: typeof probeCapabilities; + awaitCredentialSubstitution?: typeof awaitCredentialSubstitution; applyModel?: typeof applyModel; applyCodexMode?: typeof applyCodexMode; startToolRelay?: typeof startToolRelay; @@ -273,6 +275,7 @@ export interface SessionEnvironment { commitApplied: (result: { configFingerprint: string; facets: FacetDigests; + fieldDigests: Record; }) => void; plan: RunPlan; logger: Log; @@ -310,6 +313,16 @@ export interface SessionEnvironment { loadedFromContinuity: boolean; /** A remote, session-owned run whose sandbox can be parked (warm) rather than deleted at end. */ resumable: boolean; + /** + * When this sandbox's MODEL credential was delivered as a Daytona Secret (epoch millis), set + * only on a fresh create. Undefined for a plaintext-env run, a reconnect, and every local run. + * + * Read by the 401 classifier: Daytona applies a new Secret's substitution rule asynchronously, + * so a refusal shortly after delivery is the propagation race rather than a bad key. The + * timestamp is what keeps that reading honest — a warm sandbox's turn an hour later gets the + * ordinary auth advice, because by then the placeholder explanation is no longer available. + */ + modelSecretDeliveredAt?: number; /** The conversation turn index this acquire's continuity record was read/written at. */ continuityTurnIndex: number | undefined; // Mutable teardown/turn state shared across acquire, runTurn, and destroy. @@ -387,4 +400,14 @@ export interface SessionEnvironment { export type AcquireEnvironmentResult = | { ok: true; env: SessionEnvironment } - | { ok: false; error: string }; + | { + ok: false; + error: string; + /** + * The preflight proved this sandbox never got its Secret substitution wiring (the fault + * is binary per sandbox and permanent — see credential-preflight.ts). The environment is + * already destroyed; the acquire wrapper retries once with a fresh sandbox, because a new + * sandbox on the same Secret works. + */ + stuckSubstitution?: boolean; + }; diff --git a/services/runner/src/engines/sandbox_agent/session-continuity.ts b/services/runner/src/engines/sandbox_agent/session-continuity.ts index 684c00b539..7a6c7e3902 100644 Binary files a/services/runner/src/engines/sandbox_agent/session-continuity.ts and b/services/runner/src/engines/sandbox_agent/session-continuity.ts differ diff --git a/services/runner/src/engines/sandbox_agent/session-identity.ts b/services/runner/src/engines/sandbox_agent/session-identity.ts index 54ccde6d89..50897e1fad 100644 --- a/services/runner/src/engines/sandbox_agent/session-identity.ts +++ b/services/runner/src/engines/sandbox_agent/session-identity.ts @@ -12,7 +12,7 @@ import { userTurnCarriesContent, } from "../../protocol.ts"; import { approvalDecisionOf } from "../../responder.ts"; -import { resolveCodexMode } from "./codex-mode.ts"; +import { normalizedHarnessMode } from "../../harness-kind.ts"; import type { TeardownReason } from "./teardown.ts"; import { loadRunnerConfig } from "../../config/runner-config.ts"; @@ -211,18 +211,70 @@ function canonicalJson(value: unknown): string { * sandbox that was still perfectly usable, which is the exact cost this project exists to remove. * They stay in `runContext` for tool binding and observability; they simply no longer decide * whether an environment may be reused. + * + * One `runContext` field is deliberately IN the hash although the object as a whole is + * excluded: `workflow.artifact.id`, because the agent mount it selects is baked at acquire + * (audit finding 4; see the field comment in `configShape`). + * + * `modelCapabilities` left the hash the same way (2026-08-30, cold/warm audit finding 2). It is + * the resolved model's input modalities, read ONLY by the per-turn attachment-delivery chain — + * nothing bakes it into the environment. And because it changes WITH the model, hashing it made + * the plan for a vision-to-text model switch move a second facet, so the live `setModel` route + * was refused and the switch rebuilt the sandbox. The model id itself stays in the hash; its + * per-turn side facts do not. */ export function configFingerprint(request: AgentRunRequest): string { + return sha256(canonicalJson(configShape(request))); +} + +/** + * Per-field digests of the SAME shape `configFingerprint` hashes, so a config mismatch can name + * WHICH fields differ (names only, never values — each digest is a hash). Today a + * `mismatch (config)` eviction says a rebuild happened but not why, and answering "what changed" + * takes production access; with the applied side storing these, the eviction log names the + * fields. Sharing `configShape` keeps the two views incapable of drifting. + */ +export function configFieldDigests( + request: AgentRunRequest, +): Record { + return Object.fromEntries( + Object.entries(configShape(request)).map(([field, value]) => [ + field, + sha256(canonicalJson(value)), + ]), + ); +} + +/** The fields whose digests differ, in shape order. Empty when `applied` is absent (unknowable). */ +export function changedConfigFields( + desired: Record, + applied: Record | undefined, +): string[] { + if (!applied) return []; + return Object.keys(desired).filter( + (field) => desired[field] !== applied[field], + ); +} + +function configShape(request: AgentRunRequest) { const shape = { harness: request.harness ?? null, sandbox: request.sandbox ?? null, + // The ONE `runContext` field that is environment identity (audit finding 4). The agent + // artifact id signs the agent mount, mounts an artifact-keyed store prefix, sets the + // mount env var, and selects the durable-storage guidance — all baked at acquire. Without + // it a warm sandbox kept serving a session whose storage folder had changed, with the + // wrong (or no) agent mount attached. The REST of `runContext` stays out: revision ids, + // variant identity, and trace identity are per-turn metadata (the step-1 rule). + agentArtifactId: request.runContext?.workflow?.artifact?.id?.trim() || null, model: request.model ?? null, - // Harness mode is applied once, at session acquire (codex-mode.ts). Normalize Codex defaults - // and ignore the field for other harnesses so only effective mode changes evict warm sessions. - harnessMode: - request.harness === "codex" - ? resolveCodexMode(request.harnessMode) - : null, + // Harness mode is applied once, at session acquire (codex-mode.ts). The SHARED normalizer + // (audit findings 3 and 6) resolves Codex defaults and ignores the field elsewhere, and the + // facet digest uses the same call, so the two views can never disagree about a mode change. + harnessMode: normalizedHarnessMode(request.harness, request.harnessMode), + // Hashed on EVERY harness, deliberately (audit finding 8 proposed scoping this to Pi and + // was declined): design Decision 7 pins that a custom provider identity change cold-starts + // rather than reusing a mismatched live session, and the pinned test covers non-Pi too. connection: request.connection ?? null, modelConnection: request.modelConnection ? { @@ -239,7 +291,6 @@ export function configFingerprint(request: AgentRunRequest): string { ), } : null, - modelCapabilities: request.modelCapabilities ?? null, agentsMd: request.agentsMd ?? null, systemPrompt: request.systemPrompt ?? null, appendSystemPrompt: request.appendSystemPrompt ?? null, @@ -252,19 +303,30 @@ export function configFingerprint(request: AgentRunRequest): string { ...server, connection: { ...server.connection, + // `?? []` matches the facet digest's normalization (`credentialShapes`): an omitted + // array and an empty one are the same configuration, and the two identity views + // must agree on that or a no-op request cold-evicts with a DISAGREE log. credentials: server.connection?.credentials?.map((credential) => ({ binding: credential.binding, usage: credential.usage, - })), + })) ?? [], }, })) ?? null, - toolCallbackEndpoint: request.toolCallback?.endpoint ?? null, + // No `toolCallback.endpoint` (audit finding 5): every turn reads the INCOMING request's + // callback (`run-turn.ts` builds each dispatch from it), nothing bakes the endpoint into + // the environment, and hashing it evicted a warm session when the per-deployment gateway + // URL moved. The endpoint's per-turn AUTHORIZATION was already excluded. + // No `gatewayGuidance` and no `gatewayPolicy`: both are DERIVED from the agent's gateway + // connections at resolve time. The guidance is spliced into the prompt at environment build + // (`buildRunPlan`) and its wording treats the integration names as examples, so a warm + // session serving a slightly stale list is honest — and hashing it would evict a warm + // session every time an integration is added, the exact cost this exclusion removes. permissions: request.permissions ?? null, sandboxPermission: request.sandboxPermission ?? null, harnessFiles: request.harnessFiles ?? null, // No `workflowRevision` and no `isDraft`. See the doc comment above. }; - return sha256(canonicalJson(shape)); + return shape; } function collectToolCallIds( diff --git a/services/runner/src/environment/apply-plan.ts b/services/runner/src/environment/apply-plan.ts index b960d55a66..b1b0d94528 100644 --- a/services/runner/src/environment/apply-plan.ts +++ b/services/runner/src/environment/apply-plan.ts @@ -33,8 +33,7 @@ import type { AgentRunRequest } from "../protocol.ts"; import { applyModel } from "../engines/sandbox_agent/model.ts"; import { resolveSkillDirs } from "../engines/skills.ts"; import type { SessionEnvironment } from "../engines/sandbox_agent/runtime-contracts.ts"; -import { normalizeDesiredState } from "../lifecycle/desired-state.ts"; -import { configFingerprint } from "../engines/sandbox_agent/session-identity.ts"; +import { appliedResultForRequest } from "../engines/sandbox_agent/applied-state.ts"; import type { ReconcilePlan } from "../lifecycle/reconcile-plan.ts"; import { refresh, type WorkspaceInventory } from "./workspace-manager.ts"; import { carriesMinimalHistory } from "../engines/sandbox_agent/session-identity.ts"; @@ -89,7 +88,9 @@ export async function applyReconcilePlan( // No inventory means this environment never recorded what it wrote, so a refresh // cannot know what to delete. Refuse rather than write-without-deleting: a stale skill // left readable is the failure this route exists to prevent. - log("live-route: no workspace inventory recorded; cannot refresh safely"); + log( + "live-route: no workspace inventory recorded; cannot refresh safely", + ); return false; } // The desired content comes from the INCOMING request. Reading it from `env.plan` would @@ -162,7 +163,9 @@ export async function applyReconcilePlan( // the conversation could not survive, so a refusal leaves the live session running and // the caller rebuilds from a clean state. if (!env.reopenSession) { - log("live-route: this environment cannot reopen its session; rebuilding"); + log( + "live-route: this environment cannot reopen its session; rebuilding", + ); return false; } const result = await env.reopenSession({ @@ -178,6 +181,17 @@ export async function applyReconcilePlan( } case "apply-live": { + if (action.facet !== "model") { + // `apply-live` names an OPERATION, not a target: the plan says which facet asked for + // it, and the model is the only one this applier knows how to install live. A future + // plan that routes another facet here (the credential plan is the expected first) must + // fail into a rebuild, not have its change silently installed as a model (audit + // finding 7). + log( + `live-route: no live applier for facet '${action.facet}'; rebuilding`, + ); + return false; + } // The only live session-level operation: `setModel` on the running session. Strict, so a // model the harness will not accept throws here and the whole plan fails rather than // silently leaving the session on its previous model while we report the new one. @@ -207,11 +221,7 @@ export async function applyReconcilePlan( // EVERY action succeeded. Only now may the environment claim the new configuration, and this // is the single call that lets it. See "THE ONE RULE" above. - const fingerprint = configFingerprint(request); - env.commitApplied({ - configFingerprint: fingerprint, - facets: normalizeDesiredState(request, fingerprint).digests, - }); + env.commitApplied(appliedResultForRequest(request)); return true; } diff --git a/services/runner/src/environment/timing.ts b/services/runner/src/environment/timing.ts index f204b0a8a1..a454044231 100644 --- a/services/runner/src/environment/timing.ts +++ b/services/runner/src/environment/timing.ts @@ -38,6 +38,7 @@ export const ACQUIRE_STAGES = [ "prepare_workspace", "probe_capabilities", "create_session", + "credential_preflight", "acquire_total", ] as const; diff --git a/services/runner/src/harness-kind.ts b/services/runner/src/harness-kind.ts new file mode 100644 index 0000000000..d675045b2d --- /dev/null +++ b/services/runner/src/harness-kind.ts @@ -0,0 +1,55 @@ +/** + * THE one harness-identity normalizer (cold/warm audit findings 3 and 6). + * + * The wire carries `pi_core` and `pi_agenta` (a removed experiment's spelling, still read for + * old stored configs), `claude`, and `codex`; an empty or absent harness defaults to `pi_core`. + * Three call sites used to re-derive that mapping with their own inline literals, and one of + * them (`reconciliation-router.harnessKind`) matched the bare "pi" the wire never carries — so + * every playground Pi run fell into the fail-closed all-rebuild capability row (#6364). A second + * drift (finding 3): the config fingerprint normalized the Codex harness mode while the facet + * digest took it raw, so the two views could disagree about whether anything changed. + * + * This module exists so that class of drift is unrepresentable: everything that asks "which + * harness family is this?" or "what is the effective harness mode?" asks HERE, and a round-trip + * test pins every wire spelling. Keep it dependency-light (one pure import) so any layer — + * engines, lifecycle, tracing — can use it without cycles. + */ +import { resolveCodexMode } from "./engines/sandbox_agent/codex-mode.ts"; + +/** The harness FAMILY a wire spelling resolves to. `unknown` must always fail closed. */ +export type NormalizedHarnessKind = "pi" | "claude" | "codex" | "unknown"; + +/** Every wire spelling this runner accepts, mapped to its family. */ +export function harnessKindOf( + harness: string | undefined, +): NormalizedHarnessKind { + // Only an ABSENT or EMPTY harness takes the `pi_core` default. `/stream` decodes its body + // with an unchecked `JSON.parse(raw) as AgentRunRequest`, so a malformed payload can put + // `null`, `0`, or `false` in this field, and a bare `||` would hand each of them Pi's live + // routes. A non-string is not a harness spelling we recognize, so it fails closed. The real + // client always sends a string (`wire.py` writes `harness.value`), so this costs a wasted + // rebuild only on input that should not exist. + if (harness !== undefined && typeof harness !== "string") return "unknown"; + const resolved = harness || "pi_core"; + if (resolved === "pi" || resolved === "pi_core" || resolved === "pi_agenta") + return "pi"; + if (resolved === "claude" || resolved === "codex") return resolved; + return "unknown"; +} + +/** + * The EFFECTIVE harness mode: what session acquire will actually apply. + * + * Codex normalizes through `resolveCodexMode` (an invalid or absent value becomes the default), + * and every other harness has no mode at all — so an explicitly-sent default, an absent field, + * and a mode on a harness that ignores it all normalize to the same value, and only a change + * the session would OBSERVE can move a fingerprint or a facet. + */ +export function normalizedHarnessMode( + harness: string | undefined, + harnessMode: string | undefined, +): string | null { + return harnessKindOf(harness) === "codex" + ? resolveCodexMode(harnessMode) + : null; +} diff --git a/services/runner/src/lifecycle/desired-state.ts b/services/runner/src/lifecycle/desired-state.ts index 6b8e6d4c5d..5e54fbcb66 100644 --- a/services/runner/src/lifecycle/desired-state.ts +++ b/services/runner/src/lifecycle/desired-state.ts @@ -26,6 +26,7 @@ import { createHash } from "node:crypto"; import type { AgentRunRequest } from "../protocol.ts"; +import { normalizedHarnessMode } from "../harness-kind.ts"; /** * The facets, in the order the reconciliation plan must apply them. @@ -81,9 +82,14 @@ function canonical(value: unknown): string { /** Strip credential VALUES, keeping only the shape. Mirrors `configFingerprint`. */ function credentialShapes( - credentials: ReadonlyArray<{ binding?: unknown; usage?: unknown }> | undefined, + credentials: + | ReadonlyArray<{ binding?: unknown; usage?: unknown }> + | undefined, ): unknown { - return (credentials ?? []).map((c) => ({ binding: c.binding, usage: c.usage })); + return (credentials ?? []).map((c) => ({ + binding: c.binding, + usage: c.usage, + })); } /** @@ -105,6 +111,11 @@ export function normalizeDesiredState( provider: request.sandbox ?? null, harness: request.harness ?? null, sandboxPermission: request.sandboxPermission ?? null, + // The agent artifact id lives here because the mount it selects is created with the + // sandbox and has no live remount route: a changed (or newly present) id can only be + // served by a rebuild (audit finding 4). The rest of `runContext` stays out of every + // facet — it is per-turn metadata. + agentArtifactId: request.runContext?.workflow?.artifact?.id?.trim() || null, }); // RUNTIME: what is baked into the agent daemon at start. Model connection, process @@ -133,7 +144,9 @@ export function normalizeDesiredState( skills: request.skills ?? null, }); - // PROMPTS: the system and append prompts. + // PROMPTS: the system and append prompts. `gatewayGuidance` is deliberately absent here and + // from every other facet (mirroring `configFingerprint`): it is derived config the runner + // splices at build time, refreshed by whatever rebuild happens anyway, never a reason for one. // // SEPARATE FROM `workspaceFiles`, and not live. For Pi these land as files under the agent // directory, and the adapter matrix records active-session observation as NOT GUARANTEED: a @@ -169,9 +182,14 @@ export function normalizeDesiredState( // own facet: section 1.4 exempts permission TIGHTENING from apply-live entirely, and it must // take effect or fail closed before execution continues. `mcpServers` is here because the // server LIST is only read at session initialization and no live API exists on any harness. + // No `modelCapabilities`: it is per-turn data (the attachment chain reads the incoming + // request every turn) and it changes WITH the model, so hashing it here made a cross-modality + // model switch move `harnessSession` beside `model` and refuse the live route (audit finding 2). const harnessSession = canonical({ - harnessMode: request.harnessMode ?? null, - modelCapabilities: request.modelCapabilities ?? null, + // The SHARED normalizer, not the raw field (audit finding 3): the fingerprint normalizes + // the Codex mode, and a facet that took it raw could move while the fingerprint stayed + // still — poisoning every later plan for that session into a rebuild. + harnessMode: normalizedHarnessMode(request.harness, request.harnessMode), permissions: request.permissions ?? null, mcpServers: request.mcpServers?.map((server) => ({ @@ -185,9 +203,10 @@ export function normalizeDesiredState( // TOOL CATALOG: what the model can see and call. It is its own facet because it is the one // the adapters could eventually apply live. v1 routes it to a session reopen on every harness. + // No `toolCallback.endpoint` (audit finding 5): it is read from the incoming request every + // turn, so it is per-turn routing, not environment identity. const toolCatalog = canonical({ customTools: request.customTools ?? null, - toolCallbackEndpoint: request.toolCallback?.endpoint ?? null, }); return { diff --git a/services/runner/src/lifecycle/reconciliation-router.ts b/services/runner/src/lifecycle/reconciliation-router.ts index 1c651ea2c0..26336d8ffa 100644 --- a/services/runner/src/lifecycle/reconciliation-router.ts +++ b/services/runner/src/lifecycle/reconciliation-router.ts @@ -29,7 +29,9 @@ import { type ReconcilePlan, } from "./reconcile-plan.ts"; -export type HarnessKind = "pi" | "claude" | "codex" | "unknown"; +import { harnessKindOf, type NormalizedHarnessKind } from "../harness-kind.ts"; + +export type HarnessKind = NormalizedHarnessKind; /** * What each harness can do about a changed facet. @@ -220,10 +222,13 @@ const V1_CAPABILITIES: Readonly< }; export function harnessKind(request: AgentRunRequest): HarnessKind { - const harness = request.harness; - if (harness === "pi" || harness === "claude" || harness === "codex") - return harness; - return "unknown"; + // Delegates to THE one normalizer (audit finding 6). This function's first version matched + // the bare "pi" literal the wire never carries, sending every playground Pi run into the + // fail-closed all-rebuild row (#6364) — the shared normalizer plus its round-trip test is + // what makes that drift unrepresentable now. The non-string fail-closed guard (an unchecked + // `JSON.parse` can put `null`, `0`, or `false` here) lives inside the normalizer too, so + // every caller gets it. + return harnessKindOf(request.harness); } export function capabilitiesFor( diff --git a/services/runner/src/lifecycle/session-coordinator.ts b/services/runner/src/lifecycle/session-coordinator.ts index 3402ee58d5..4ab7ca810d 100644 --- a/services/runner/src/lifecycle/session-coordinator.ts +++ b/services/runner/src/lifecycle/session-coordinator.ts @@ -40,7 +40,10 @@ import { type SessionEnvironment, } from "../engines/sandbox_agent.ts"; import type { MountCredentials } from "../engines/sandbox_agent/mount.ts"; -import type { TeardownReason } from "../engines/sandbox_agent/teardown.ts"; +import { + teardownDisposition, + type TeardownReason, +} from "../engines/sandbox_agent/teardown.ts"; import { mechanismForRotation, runCredentialDelivery, @@ -52,7 +55,9 @@ import { desiredCredentialSetFor } from "../providers/daytona-credential-deliver import { approvalDecisionForToolCall, assertsPriorConversation, + changedConfigFields, computeCredentialEpoch, + configFieldDigests, configFingerprint, credentialEpochMismatch, carriesApprovalReplyOnly, @@ -593,6 +598,26 @@ export async function runWithKeepalive( return "runtime-incompatible"; }; + /** + * The teardown for a set of unresolved reasons: the strictest one wins. + * + * An eviction is named by its FIRST unresolved reason, but the sandbox's fate must answer + * ALL of them. `history` sorts before the credential checks, so a turn that changed the + * model, edited its transcript, AND carried a rotation the port could not deliver was + * evicted as `history`, mapped to `continuity-invalid`, and PARKED — leaving a sandbox whose + * daemon still held the old credential for the next turn to resume onto. Deleting when any + * reason says delete costs a rebuild; parking when one says delete is the stale-material bug + * `teardown.ts` exists to prevent. + */ + const strictestTeardown = (mismatches: string[]): TeardownReason => { + const reasons = mismatches.map(mismatchTeardownReason); + return ( + reasons.find((r) => teardownDisposition(r) === "delete") ?? + reasons[0] ?? + "compatibility-mismatch" + ); + }; + const notifyParkedLive = async (env: SessionEnvironment): Promise => { if (resolveKeepaliveProvider(request) !== "daytona") return; // Best-effort: the session is already parked, so an activity-refresh failure must not turn @@ -827,7 +852,12 @@ export async function runWithKeepalive( // where `reserve` would) means unmounting and deleting that cwd out from under the environment // just built on it. That is the original bug in a narrower window, so the claim happens here. await pool.evict(key, "pre-acquire", "failed-turn"); - const acq = await engine.acquireEnvironment(request, signal, signed, trackedEmit); + const acq = await engine.acquireEnvironment( + request, + signal, + signed, + trackedEmit, + ); if (!acq.ok) return { ok: false, error: acq.error }; const env = acq.env; const leaseMs = installedMountLease(env.installedMountExpiries); @@ -898,22 +928,49 @@ export async function runWithKeepalive( // Comparing anyway evicts the warm session on every turn of every conversation. The session // id already binds the request to this conversation; the client simply no longer asserts it. const clientAssertsHistory = !carriesMinimalHistory(request); - let mismatch: string | undefined; - if (cfgFp !== existing.configFingerprint) mismatch = "config"; - else if (clientAssertsHistory && priorFp !== existing.historyFingerprint) - mismatch = "history"; - else if (credMismatch) mismatch = credMismatch; - else if ( - // Still-valid credentials that cannot cover a worst-case turn are expiring, not expired: - // rebuild at the boundary rather than let the turn die under the mount. - mountCredentialsExpireBy(existing.credentialEpoch, requiredValidThroughMs) - ) - mismatch = "credentials-expiring"; - else if (!tailIsFreshUserMessage(request)) mismatch = "tail"; + /** + * EVERY reason is re-evaluated after a repair, not only the first one found. + * + * The old shape was an else-if chain feeding the two repair doors below, and a successful + * repair set `mismatch = undefined`. That cleared the answer to EVERY question when the + * repair had answered exactly one: a model switch riding with an edited transcript took the + * live route and then continued warm on a native conversation that still held the unedited + * turn; paired with a rotated credential it ran on the old baked key; paired with an + * expiring mount lease it let the turn die under the mount. So a repair marks ITS reason + * repaired and asks again, and the remaining checks keep their order and their comments. + */ + const repaired = new Set(); + /** + * Every unresolved reason, in the order the checks are written. The FIRST one names the + * eviction, and the WHOLE list decides the teardown — see `strictestTeardown`. + */ + const unresolvedMismatches = (): string[] => { + const reasons: string[] = []; + if (!repaired.has("config") && cfgFp !== existing.configFingerprint) + reasons.push("config"); + if (clientAssertsHistory && priorFp !== existing.historyFingerprint) + reasons.push("history"); + if (credMismatch && !repaired.has(credMismatch)) + reasons.push(credMismatch); + if ( + // Still-valid credentials that cannot cover a worst-case turn are expiring, not expired: + // rebuild at the boundary rather than let the turn die under the mount. + mountCredentialsExpireBy( + existing.credentialEpoch, + requiredValidThroughMs, + ) + ) + reasons.push("credentials-expiring"); + if (!tailIsFreshUserMessage(request)) reasons.push("tail"); + return reasons; + }; + const firstMismatch = (): string | undefined => unresolvedMismatches()[0]; + let mismatch = firstMismatch(); // STEP 6. A pure configuration mismatch gets one chance to be satisfied on the live // environment. Everything else — credentials, continuity, an expiring lease — is decided - // above and never reaches this door. + // by `firstMismatch` and never reaches this door; a successful apply answers ONLY the + // config question, so the remaining reasons are asked again. if (mismatch === "config") { // Pass the plan we ACTED ON: the apply has already committed the new applied state, so // recomputing here would yield an empty plan and the counter could not name the route. @@ -926,13 +983,15 @@ export async function runWithKeepalive( "environment", appliedPlan, ); - mismatch = undefined; + repaired.add("config"); + mismatch = firstMismatch(); } } // STEP 8. A rotated credential gets the same chance. The other credential mismatches do NOT: // `credentials-expired` and `credentials-expiring` are mount-lease facts that the mount // subsystem repairs by re-signing, and delivering a model key would not extend a lease. + // Same contract as step 6: a delivery answers only the rotation, so ask again. if (mismatch === "credentials-rotated") { const deliveredPlan = await tryCredentialRoute(existing); if (deliveredPlan) { @@ -944,7 +1003,8 @@ export async function runWithKeepalive( deliveredPlan, "rotate-in-place", ); - mismatch = undefined; + repaired.add("credentials-rotated"); + mismatch = firstMismatch(); } } @@ -955,14 +1015,39 @@ export async function runWithKeepalive( mismatch = "mount-lost"; if (mismatch) { - klog(`mismatch (${mismatch}) key=${key}; evict + cold`); + // The eviction is NAMED by the first unresolved reason and DISPOSED by all of them. The + // backstop only fires when the list is already empty, so `mount-lost` stands alone. + const allMismatches = + mismatch === "mount-lost" ? ["mount-lost"] : unresolvedMismatches(); + // A config mismatch names the changed FIELDS (names only — the digests never carry + // values), so "what evicted this warm session" is answerable from the log line alone + // instead of needing production database access to reconstruct. + const changedFields = + mismatch === "config" + ? changedConfigFields( + configFieldDigests(request), + existing.environment.appliedState.fieldDigests, + ) + : []; + klog( + `mismatch (${mismatch}) key=${key}` + + (changedFields.length ? ` fields=[${changedFields.join(",")}]` : "") + + // Name the rest too: they do not name the eviction but they DO decide the teardown, + // so a parked-vs-deleted sandbox is explainable from this one line. + (allMismatches.length > 1 + ? ` also=[${allMismatches.slice(1).join(",")}]` + : "") + + `; evict + cold`, + ); // A transcript mismatch is a decision about the CONVERSATION, not the environment, so it - // is logged but never counted against the router. See `DecisionScope`. + // is logged but never counted against the router. See `DecisionScope`. A continuity reason + // riding WITH an environment reason is an environment decision: the router must see the + // rebuild it is accountable for. shadowRoute( existing, "rebuild", `mismatch:${mismatch}`, - mismatch === "history" || mismatch === "tail" + allMismatches.every((r) => r === "history" || r === "tail") ? "continuity" : "environment", undefined, @@ -988,8 +1073,9 @@ export async function runWithKeepalive( `mismatch:${mismatch}`, // A failed delivery brings its own teardown reason, so the disposition travels with the // failure instead of being re-derived from a label. They agree today; the point is that - // they cannot drift. - credentialTeardown ?? mismatchTeardownReason(mismatch), + // they cannot drift. Otherwise every unresolved reason gets a vote and the strictest + // wins, so a continuity reason that merely sorts first cannot park a stale sandbox. + credentialTeardown ?? strictestTeardown(allMismatches), ); return coldAndPark(); } diff --git a/services/runner/src/protocol.ts b/services/runner/src/protocol.ts index 41066698e7..4e13fcba14 100644 --- a/services/runner/src/protocol.ts +++ b/services/runner/src/protocol.ts @@ -615,8 +615,9 @@ export interface GatewayPolicy { export interface AgentRunRequest { /** - * Harness id: "pi_core" | "pi_agenta" | "claude". `pi_core` and `pi_agenta` both drive the - * ACP agent "pi" (pi_agenta is Pi with Agenta's forced skills/prompt/policy); "claude" drives + * Harness id: "pi_core" | "claude" | "codex". `pi_core` drives the ACP agent "pi"; + * "pi_agenta" (a removed 2026 experiment) is still read as `pi_core` so an old + * stored request replays. "claude" drives * the ACP agent "claude". Selected by the request; there is no engine selector. */ harness?: string; @@ -701,6 +702,21 @@ export interface AgentRunRequest { * yet. */ gatewayPolicy?: GatewayPolicy; + /** + * The derived gateway-tools instruction section (how to use `search_tools` / `run_tool`, + * with the configured integration names as EXAMPLES), plus which prompt surface carries it. + * + * Its own field, deliberately OUTSIDE `configFingerprint` and the desired-state facets: the + * text is derived from the agent's connections at resolve time, and the runner splices it + * into `carrier` when it BUILDS an environment (`buildRunPlan`). So adding or removing an + * integration never evicts a warm session for a one-word prompt change; the names refresh + * on the next session build, and the wording says the list may be stale. When it was + * composed into the prompt strings upstream, every integration add went cold. + */ + gatewayGuidance?: { + text: string; + carrier: "appendSystemPrompt" | "agentsMd"; + }; /** * The declared sandbox security boundary (Layer 2). Omitted when unset. The network policy is * enforced on Daytona; on the local sidecar a restricted-network run is rejected under diff --git a/services/runner/src/sessions/attachments.ts b/services/runner/src/sessions/attachments.ts index dcb2bac249..3ee9e942a0 100644 --- a/services/runner/src/sessions/attachments.ts +++ b/services/runner/src/sessions/attachments.ts @@ -109,7 +109,11 @@ export async function fetchAttachment( headers: { authorization: auth() }, signal: AbortSignal.timeout(attachmentFetchTimeoutMs()), }); - if (!response.ok) throw new Error(`HTTP ${response.status}`); + // Prefixed so the run-turn classifier can tell a RUNNER-side 401 from the provider's. + // A bare `HTTP 401` here is indistinguishable from a model refusal, and the credential-race + // branch would then spend the session's one report on an attachment fetch. + if (!response.ok) + throw new Error(`attachment fetch failed: HTTP ${response.status}`); // Content-Type parameters (charset, boundary) are not part of the MIME identity the // capability gate and the allowlist compare on, so keep the bare type. @@ -161,7 +165,8 @@ export async function claimAttachments( }), signal: AbortSignal.timeout(attachmentFetchTimeoutMs()), }); - if (!response.ok) throw new Error(`HTTP ${response.status}`); + if (!response.ok) + throw new Error(`attachment claim failed: HTTP ${response.status}`); return true; } catch (error) { log( diff --git a/services/runner/src/sessions/persist.ts b/services/runner/src/sessions/persist.ts index 8a67ebbf88..56ac40dc24 100644 --- a/services/runner/src/sessions/persist.ts +++ b/services/runner/src/sessions/persist.ts @@ -45,16 +45,24 @@ const DURABLE_INGEST_MAX_RETRIES_CAP = 12; * mean on — the compose files pass the var through as `${AGENTA_RECORDS_DURABLE:-}`, which sets * an empty string when unset. "false" → the fire-and-forget legacy path, unchanged. */ function durableRecordsEnabled(): boolean { - return String(process.env.AGENTA_RECORDS_DURABLE ?? "").trim().toLowerCase() !== "false"; + return ( + String(process.env.AGENTA_RECORDS_DURABLE ?? "") + .trim() + .toLowerCase() !== "false" + ); } /** Attempts before a durable-mode drop; env-overridable for ops tuning (and fast tests). */ function durableMaxRetries(): number { - return envInt("AGENTA_RECORDS_INGEST_MAX_RETRIES", DURABLE_INGEST_MAX_RETRIES, { - min: 1, - max: DURABLE_INGEST_MAX_RETRIES_CAP, - log, - }); + return envInt( + "AGENTA_RECORDS_INGEST_MAX_RETRIES", + DURABLE_INGEST_MAX_RETRIES, + { + min: 1, + max: DURABLE_INGEST_MAX_RETRIES_CAP, + log, + }, + ); } function log(msg: string): void { @@ -108,7 +116,10 @@ async function postEvent( ...(spanId ? { span_id: spanId } : {}), }), }); - if (!res.ok) throw new Error(`HTTP ${res.status}`); + // Prefixed so a runner-side 401 is distinguishable from a provider refusal; see + // `RUNNER_INTERNAL_401` in engines/sandbox_agent/errors.ts. + if (!res.ok) + throw new Error(`session records persist failed: HTTP ${res.status}`); log( `ingest OK session=${sessionId} idx=${eventIndex} type=${event.type}`, ); @@ -219,7 +230,9 @@ export function recordsIncomplete(sessionId: string): boolean { * substitute for a close signal the harness may never send (a call that streams then * stalls without a `tool_result`). */ -const OPEN_TOOL_TTL_MS = envTimerMs("AGENTA_RECORD_TOOL_TTL_MS", 3_000, { log }); +const OPEN_TOOL_TTL_MS = envTimerMs("AGENTA_RECORD_TOOL_TTL_MS", 3_000, { + log, +}); /** * Build an emitter that persists every event via the ingest chain AND calls the diff --git a/services/runner/src/sessions/records-query.ts b/services/runner/src/sessions/records-query.ts index 5792f1d711..d7713dfdce 100644 --- a/services/runner/src/sessions/records-query.ts +++ b/services/runner/src/sessions/records-query.ts @@ -48,13 +48,17 @@ export async function fetchSessionRecords( body: JSON.stringify({ session_id: sessionId }), signal: AbortSignal.timeout(queryTimeoutMs()), }); - if (!res.ok) throw new Error(`HTTP ${res.status}`); + // Prefixed so a runner-side 401 is distinguishable from a provider refusal; see + // `RUNNER_INTERNAL_401` in engines/sandbox_agent/errors.ts. + if (!res.ok) + throw new Error(`session records query failed: HTTP ${res.status}`); const body = (await res.json()) as { records?: SessionRecordRow[] }; return Array.isArray(body?.records) ? body.records : []; } catch (err) { - const detail = String( - err instanceof Error ? err.message : err, - ).slice(0, 120); + const detail = String(err instanceof Error ? err.message : err).slice( + 0, + 120, + ); log(`query FAILED session=${sessionId}: ${detail}`); return null; } diff --git a/services/runner/src/subscription-status.ts b/services/runner/src/subscription-status.ts index 01fd389d07..b9e46c2ca7 100644 --- a/services/runner/src/subscription-status.ts +++ b/services/runner/src/subscription-status.ts @@ -41,12 +41,7 @@ export type SubscriptionState = /** This runner version cannot check this harness. */ | "unsupported"; -export const SUBSCRIPTION_HARNESSES = [ - "codex", - "claude", - "pi_core", - "pi_agenta", -] as const; +export const SUBSCRIPTION_HARNESSES = ["codex", "claude", "pi_core"] as const; export type SubscriptionHarness = (typeof SUBSCRIPTION_HARNESSES)[number]; export interface HarnessSubscriptionStatus { @@ -139,10 +134,7 @@ const PROBES: Record = { file: ".credentials.json", provider: "anthropic", }, - // `pi_core` and `pi_agenta` both drive the ACP agent "pi" (see run-plan.ts), so they read the - // same login on the same mount: one probe, reported once per harness. pi_core: PI_PROBE, - pi_agenta: PI_PROBE, }; /** diff --git a/services/runner/src/tracing/otel.ts b/services/runner/src/tracing/otel.ts index b5fa907741..0f9034c146 100644 --- a/services/runner/src/tracing/otel.ts +++ b/services/runner/src/tracing/otel.ts @@ -360,14 +360,78 @@ export function isAgentaIngest(endpoint: string): boolean { ); } +/** Hosts the platform rewrites to `host.docker.internal` before it dispatches a run. */ +const BRIDGE_REWRITTEN_HOSTS = new Set(["localhost", "0.0.0.0"]); + +/** + * The same base the platform would hand a dispatched run, or undefined when it rewrites nothing. + * + * A self-hoster's natural `AGENTA_API_URL` is a localhost URL, but a sandboxed run cannot reach + * the host that way from inside its own container, so the trace endpoint is rewritten to + * `host.docker.internal` on the way out. The runner reads only the raw env value, so the + * configured base and the endpoint it is handed can never string-match, and every run of such a + * deployment loses its platform credential. + * + * The authority for that rewrite is `parse_url` in `sdks/python/agenta/sdk/utils/helpers.py`, NOT + * the api's same-named twin in `api/oss/src/utils/helpers.py`. The endpoint on the wire is + * `ag.tracing.otlp_url`, which the agent service derives through the SDK's copy; the api's copy + * shapes the service URL a run is POSTed to. The two differ, and the differences are what the + * limits below are about. + * + * The mirror is deliberately exact: same scheme, port, and path, and only the two hosts the + * rewrite touches. Three things it deliberately does NOT do: + * + * - `127.0.0.1` earns no alias. Neither copy of `parse_url` rewrites it, so that deployment + * already matches its own raw base and the bridge form is a pair the platform cannot produce. + * - A scheme-less base earns no alias, because it cannot help. The SDK's `parse_url` does no + * scheme defaulting (unlike the api's), so a scheme-less `AGENTA_API_URL` yields an equally + * scheme-less ENDPOINT, which `new URL` reads as an opaque `localhost:`-scheme path. Both + * sides are then unparseable and no aliasing here can make them agree. Such a deployment is + * broken further upstream — the OTLP exporter target itself is malformed — and the fix is + * scheme defaulting in the SDK's `parse_url`, not a wider allowlist. + * - The alias is not gated on the docker network mode, even though the rewrite is: the SDK's + * `parse_url` rewrites only when `DOCKER_NETWORK_MODE` is exactly `bridge`, so in `host` mode + * (and when the var is unset) the platform dispatches the unrewritten localhost endpoint, + * which the raw base already matches, and this alias is merely unused. Gating it is not + * possible and would not be safe: the runner's environment carries no `DOCKER_NETWORK_MODE` + * (it takes no `env_file` by design, and the var is absent from its `environment:` block in + * every compose file), so the runner cannot tell "unset" from "bridge, but invisible to me" — + * and those two need OPPOSITE answers. A mode-gated alias would read "unset" and emit + * nothing on exactly the bridge deployments this exists to fix. + * + * What that last point leaves is a residual width: in host mode the allowlist also admits + * `host.docker.internal` on the configured port, an endpoint the platform will not dispatch + * there. It is not an attack surface. The endpoint is always platform-dispatched — the runner + * reads it from the run request the agent service builds — and reaching the runner directly needs + * `AGENTA_RUNNER_TOKEN` inside the compose network, where a forged request carries its own + * credential anyway. The only real exposure is a third-party collector on the docker host at the + * exact port of the configured Agenta api, which in practice is that api. + */ +function bridgeRewrittenBase(base: string): string | undefined { + let url: URL; + try { + url = new URL(base); + } catch { + return undefined; + } + if (!BRIDGE_REWRITTEN_HOSTS.has(url.hostname)) return undefined; + url.hostname = "host.docker.internal"; + return url.toString().replace(/\/+$/, ""); +} + /** The api bases `isAgentaIngest` accepts, in precedence order. Exported so a rejection can name * what it compared against — the failure is always a configuration gap, never a code path. */ export function configuredIngestBases(): string[] { - return [ + const configured = [ process.env.AGENTA_API_INTERNAL_URL, process.env.AGENTA_API_URL, CLOUD_API_BASE, ].filter((base): base is string => Boolean(base)); + + return configured.flatMap((base) => { + const bridged = bridgeRewrittenBase(base); + return bridged ? [base, bridged] : [base]; + }); } /** diff --git a/services/runner/src/version.ts b/services/runner/src/version.ts index 185460f3fe..1c1863a3ea 100644 --- a/services/runner/src/version.ts +++ b/services/runner/src/version.ts @@ -12,7 +12,7 @@ import pkg from "../package.json"; export const PROTOCOL_VERSION = 1; export const RUNNER_VERSION: string = pkg.version; export const ENGINES = ["sandbox-agent"] as const; -export const HARNESS_KINDS = ["pi_core", "claude", "pi_agenta"] as const; +export const HARNESS_KINDS = ["pi_core", "claude"] as const; export interface RunnerInfo { status: "ok"; diff --git a/services/runner/tests/unit/credential-preflight.test.ts b/services/runner/tests/unit/credential-preflight.test.ts new file mode 100644 index 0000000000..db52ea995b --- /dev/null +++ b/services/runner/tests/unit/credential-preflight.test.ts @@ -0,0 +1,177 @@ +/** + * Unit tests for the Daytona credential-substitution preflight. + * + * Run: pnpm exec vitest run tests/unit/credential-preflight.test.ts + */ +import { describe, it } from "vitest"; +import assert from "node:assert/strict"; + +import { + awaitCredentialSubstitution, + deliversModelSecretOnCreate, + type PreflightSandbox, +} from "../../src/engines/sandbox_agent/credential-preflight.ts"; + +/** A sandbox whose probe responses play back in order (the last repeats forever). */ +function sandboxAnswering(bodies: (string | Error)[]): { + sandbox: PreflightSandbox; + commands: string[]; +} { + const commands: string[] = []; + let index = 0; + return { + commands, + sandbox: { + async runProcess(request) { + commands.push(request.args?.[1] ?? request.command); + const body = bodies[Math.min(index, bodies.length - 1)]; + index += 1; + if (body instanceof Error) throw body; + return { exitCode: 0, stdout: body }; + }, + }, + }; +} + +function harness(bodies: (string | Error)[], budgetMs = 25_000) { + const { sandbox, commands } = sandboxAnswering(bodies); + const logs: string[] = []; + let clock = 0; + const run = awaitCredentialSubstitution({ + sandbox, + baseUrl: "https://gateway.example/", + apiKeyVar: "OPENAI_API_KEY", + log: (m) => logs.push(m), + budgetMs, + pollMs: 2_000, + now: () => clock, + sleep: async (ms) => { + clock += ms; + }, + }); + return { run, logs, commands }; +} + +describe("awaitCredentialSubstitution", () => { + it("returns ok immediately when the first probe substitutes", async () => { + const { run, logs, commands } = harness([ + '{"error":{"message":"you must provide a model parameter"}}', + ]); + assert.equal(await run, "ok"); + assert.equal(commands.length, 1); + assert.deepEqual(logs, []); + // The probe expands the env var in the sandbox shell; the raw value never appears here, + // and the endpoint is the connection's own chat path. + assert.match(commands[0], /Bearer \$OPENAI_API_KEY/); + assert.match(commands[0], /https:\/\/gateway\.example\/chat\/completions/); + }); + + it("tolerates early raw echoes and returns ok once substitution shows", async () => { + const { run, logs, commands } = harness([ + "LiteLLM Virtual Key expected. Received=dtn_****9maz", + '{"error":{"message":"Received=dtn_****9maz"}}', + '{"error":{"message":"you must provide a model parameter"}}', + ]); + assert.equal(await run, "ok"); + assert.equal(commands.length, 3); + assert.match(logs[0], /raw placeholder echoed \(probe 1/); + assert.match(logs[2], /substitution confirmed after 3 probes/); + }); + + it("convicts the sandbox as STUCK once the 10s grace is spent", async () => { + // The grace is deliberately below Daytona's ~30s bound (see the module doc): every + // healthy sandbox we measured answered on its first probe, so waiting longer only + // holds a stuck user turn. The verdict is "stuck", never a fail-open pass. + const { run, logs, commands } = harness(["Received=dtn_****9maz"]); + assert.equal(await run, "stuck"); + assert.ok( + commands.length >= 4, + `expected >=4 probes inside the 10s grace, got ${commands.length}`, + ); + assert.match(logs[logs.length - 1], /STUCK: raw placeholder on all/); + }); + + it("convicts sooner when the caller passes a smaller budget", async () => { + const { run, logs } = harness(["Received=dtn_****9maz"], 3_000); + assert.equal(await run, "stuck"); + assert.match(logs[logs.length - 1], /STUCK/); + }); + + it("fails open (ok) when the exec channel itself errors", async () => { + const { run, logs, commands } = harness([new Error("daemon gone")]); + assert.equal(await run, "ok"); + assert.equal(commands.length, 1); + assert.match(logs[0], /probe errored, proceeding: daemon gone/); + }); + + it("treats an empty body as substituted (nothing to judge by)", async () => { + const { run, logs } = harness([""]); + assert.equal(await run, "ok"); + assert.deepEqual(logs, []); + }); + + it("fails open when a HEALTHY echoed key was scrubbed into a full placeholder", async () => { + // Daytona's egress proxy rewrites real credential values in responses back into + // `dtn_secret_`. An endpoint that echoes the Authorization header therefore returns + // this body on a perfectly healthy sandbox. Convicting on it destroyed both acquire + // attempts and failed a first turn whose real model call would have worked. + // The id is spelled in the message rather than as a `key` field so the secret scanner + // does not read this placeholder — the very thing that exists so no real key is here — + // as a leaked credential. + const { run, logs, commands } = harness([ + '{"error":{"message":"unauthorized bearer dtn_secret_01j9maz7q0"}}', + ]); + assert.equal(await run, "ok"); + assert.equal(commands.length, 1, "an unmasked echo must not be re-probed"); + assert.match(logs[0], /unmasked placeholder-shaped echo/); + }); + + it("convicts on a masked echo, whatever the provider's mask shape", async () => { + // Masking is what scrubbing cannot forge: the masked string no longer holds the real + // value for the scrubber to match, so `dtn_` beside a mask means the raw placeholder + // really went out. Both proven provider shapes must convict. + for (const masked of [ + "LiteLLM Virtual Key expected. Received=dtn_****9maz", + '{"error":{"message":"Incorrect API key provided: dtn_secr*****9maz"}}', + ]) { + const { run } = harness([masked], 3_000); + assert.equal(await run, "stuck", masked); + } + }); +}); + +describe("deliversModelSecretOnCreate: what arms the race guards", () => { + // The preflight gates on this AND a declared endpoint; the 401 classifier arms its + // credential-race reading on this alone. `acquireEnvironment` cannot be driven without a live + // provider, so this predicate is where that condition is actually pinned. + const base = { + isDaytona: true, + sandboxMode: "create", + hasModelSecretCandidate: true, + }; + + it("is true for a fresh Daytona sandbox whose model key rides a Secret", () => { + assert.equal(deliversModelSecretOnCreate(base), true); + }); + + it("is false on a reconnect: that sandbox already proved itself", () => { + assert.equal( + deliversModelSecretOnCreate({ ...base, sandboxMode: "reconnect" }), + false, + ); + }); + + it("is false on a local run: there is no Daytona Secret", () => { + assert.equal( + deliversModelSecretOnCreate({ ...base, isDaytona: false }), + false, + ); + }); + + it("is false for a plaintext-env run: there is no placeholder to substitute", () => { + assert.equal( + deliversModelSecretOnCreate({ ...base, hasModelSecretCandidate: false }), + false, + ); + }); +}); diff --git a/services/runner/tests/unit/credential-race-classification.test.ts b/services/runner/tests/unit/credential-race-classification.test.ts new file mode 100644 index 0000000000..3c88527883 --- /dev/null +++ b/services/runner/tests/unit/credential-race-classification.test.ts @@ -0,0 +1,586 @@ +/** + * Unit tests for the direct-provider credential race (F6). + * + * THE BUG. On a Daytona run the model key is a Daytona Secret and the sandbox holds a + * `dtn_secret_` placeholder Daytona substitutes into egress asynchronously. When the first + * model call beats that propagation, the provider refuses the raw placeholder with a 401. + * `classifyRunError` used to recognize that ONLY when the error body echoed the placeholder — which + * the litellm credits proxy does and no direct provider does. api.anthropic.com answers + * "Invalid bearer token" with no echo at all, and OpenAI's echo is masked + * ("dtn_secr***************cdef"), which no longer contains the literal `dtn_secret_`. So every + * direct-path placeholder 401 was blamed on the user's key. + * + * THE INVERSE TRAP, which is why the counter exists. A genuinely wrong key produces a byte-identical + * refusal on the direct path. Telling it to retry is a dead end: a failed turn DELETES the sandbox, + * so the retry cold-acquires, re-arms the freshness window, and gets the same advice forever. One + * report per session bounds it — the second identical failure falls through to the add-a-key copy. + * + * The provider bodies below were captured live from the real endpoints (2026-08-31) with a + * synthetic probe token; no real credential appears here. + * + * Run: pnpm exec vitest run tests/unit/credential-race-classification.test.ts + */ +import { describe, it, beforeEach } from "vitest"; +import assert from "node:assert/strict"; + +import { + classifyRunError, + CREDENTIAL_RACE_REPORTS_PER_SESSION, + withinCredentialPropagationWindow, +} from "../../src/engines/sandbox_agent/errors.ts"; +import { SessionContinuityStore } from "../../src/engines/sandbox_agent/session-continuity.ts"; +import { + enableDaytonaProvider, + piTranscriptWithError, + runSilentTurn, +} from "../utils/silent-turn.ts"; + +/** Captured live: api.anthropic.com refusing a `dtn_` bearer. Note it echoes NOTHING. */ +const ANTHROPIC_DIRECT_401 = + 'API Error: 401 {"type":"error","error":{"type":"authentication_error","message":"Invalid bearer token"}}'; + +/** Captured live: OpenAI refusing the same token, echoing it MASKED. */ +const OPENAI_DIRECT_401 = + 'API Error: 401 {"error":{"message":"Incorrect API key provided: dtn_secr***************cdef. You can find your API key at https://platform.openai.com/account/api-keys.","code":"invalid_api_key"}}'; + +/** The credits proxy, which names the placeholder outright. */ +const LITELLM_PROXY_401 = + 'API Error: 401 {"error":{"message":"Authentication Error, LiteLLM Virtual Key expected. Received=dtn_****, expected to start with sk-"}}'; + +const DELIVERY_MESSAGE = + "A temporary issue kept this run's credentials from reaching the model. Send the message again."; + +const fresh = () => true; +const notFresh = () => false; + +describe("classifyRunError: the credential race on a direct provider", () => { + it("classifies an anthropic-direct 401 as delivery when the Secret is fresh", () => { + const r = classifyRunError( + new Error(ANTHROPIC_DIRECT_401), + "claude", + "anthropic", + { connection: { deployment: "direct" }, daytonaCredentialFresh: fresh }, + ); + assert.equal(r.code, "credential_delivery_failed"); + assert.equal(r.message, DELIVERY_MESSAGE); + }); + + it("classifies an openai-direct 401 as delivery when the Secret is fresh", () => { + const r = classifyRunError( + new Error(OPENAI_DIRECT_401), + "codex", + "openai", + { + connection: { deployment: "direct" }, + daytonaCredentialFresh: fresh, + }, + ); + assert.equal(r.code, "credential_delivery_failed"); + }); + + it("still classifies the litellm proxy refusal as delivery", () => { + const r = classifyRunError( + new Error(LITELLM_PROXY_401), + "claude", + "anthropic", + { + connection: { deployment: "custom" }, + daytonaCredentialFresh: notFresh, + }, + ); + assert.equal(r.code, "credential_delivery_failed"); + }); + + it("keeps the add-a-key advice on a warm sandbox (no fresh Secret)", () => { + const r = classifyRunError( + new Error(ANTHROPIC_DIRECT_401), + "claude", + "anthropic", + { + connection: { deployment: "direct" }, + daytonaCredentialFresh: notFresh, + }, + ); + assert.equal(r.code, "runner_error"); + assert.match(r.message, /add the project's Anthropic key/); + }); + + it("keeps the add-a-key advice for a run with no Daytona Secret at all", () => { + const r = classifyRunError( + new Error(ANTHROPIC_DIRECT_401), + "claude", + "anthropic", + { connection: { deployment: "direct" } }, + ); + assert.equal(r.code, "runner_error"); + assert.match(r.message, /add the project's Anthropic key/); + }); + + it("does not let a fresh Secret reclassify a non-auth failure", () => { + const r = classifyRunError( + new Error("ETIMEDOUT: sandbox create timed out after 120s"), + "claude", + "anthropic", + { daytonaCredentialFresh: fresh }, + ); + assert.equal(r.code, "runner_error"); + assert.doesNotMatch(r.message, /credentials from reaching the model/); + }); + + it("does not let a fresh Secret swallow a real credits refusal", () => { + // The budget branch is more specific and runs first: a spent key is not a delivery fault, + // and telling the user to retry would loop them against an empty balance. + const r = classifyRunError( + new Error("budget_exceeded: Crossed spend within budget"), + "claude", + "anthropic", + { daytonaCredentialFresh: fresh }, + ); + assert.equal(r.code, "starter_credits_exhausted"); + }); +}); + +describe("PLACEHOLDER_CREDENTIAL: the widened masked-echo signature", () => { + // Body-only detection, with no freshness signal at all — this is what makes the direct OpenAI + // path self-diagnosing again rather than depending on the timing window. + const byBodyAlone = (raw: string) => + classifyRunError(new Error(raw), "codex", "openai").code; + + it("matches OpenAI's masked echo", () => { + assert.equal(byBodyAlone(OPENAI_DIRECT_401), "credential_delivery_failed"); + }); + + it("matches a real mask of any width", () => { + assert.equal( + byBodyAlone("401 Incorrect API key provided: dtn_secret_ab*****"), + "credential_delivery_failed", + ); + assert.equal( + byBodyAlone("401 Incorrect API key provided: dtn_abcd***"), + "credential_delivery_failed", + ); + }); + + it("never matches an ordinary user key, masked or not", () => { + // The signature is anchored on Daytona's `dtn_` placeholder prefix, which cannot occur in a + // provider key. A masked real key must still read as an ordinary auth failure. + for (const raw of [ + "401 Incorrect API key provided: sk-proj*************abcd", + "401 Incorrect API key provided: sk-ant-api03-****", + "401 invalid api key", + "401 Unauthorized", + ]) { + assert.equal(byBodyAlone(raw), "runner_error", raw); + } + }); + + it("never matches a literal glob, which is ordinary text", () => { + // `dtn_*` is a perfectly normal thing to find in a path, a filter or a log line, and the + // first version of this signature allowed a zero-length stem, so it matched. A real mask is + // many characters wide behind a real stem. + for (const raw of [ + "401 Unauthorized while listing dtn_* secrets", + "401 Unauthorized: no match for pattern dtn_*", + "401 Unauthorized: dtn_**", + "401 Unauthorized: dtn_ab***", + ]) { + assert.equal(byBodyAlone(raw), "runner_error", raw); + } + }); + + it("needs auth context, so a masked token in an unrelated error is not a delivery fault", () => { + // A hypothetical customer key spelled `dtn_customer_***`. Inside a credential refusal it + // reads as delivery; inside anything else it must not, because the masked-echo pattern is a + // guess about formatting rather than a quoted protocol string. + assert.equal( + byBodyAlone("ETIMEDOUT while syncing dtn_customer_*** to the store"), + "runner_error", + ); + assert.equal( + byBodyAlone("failed to parse config value dtn_customer_***"), + "runner_error", + ); + assert.equal( + byBodyAlone("401 Unauthorized: dtn_customer_***"), + "credential_delivery_failed", + ); + }); + + it("keeps the two self-evidencing signatures free of the auth requirement", () => { + // Those name the placeholder in a shape only the delivery layer produces, so they carry + // their own proof and must not be weakened by the corroboration rule above. + assert.equal( + byBodyAlone("tool run failed: dtn_secret_abc123 was rejected downstream"), + "credential_delivery_failed", + ); + }); +}); + +describe("withinCredentialPropagationWindow", () => { + const now = 1_000_000; + + it("is false when no Daytona Secret was delivered", () => { + assert.equal(withinCredentialPropagationWindow(undefined, now), false); + }); + + it("is true just inside the window (59s)", () => { + assert.equal(withinCredentialPropagationWindow(now - 59_000, now), true); + }); + + it("is false just outside the window (61s)", () => { + assert.equal(withinCredentialPropagationWindow(now - 61_000, now), false); + }); +}); + +describe("the once-per-session bound (the inverse failure mode)", () => { + let store: SessionContinuityStore; + + beforeEach(() => { + store = new SessionContinuityStore(); + }); + + /** The predicate `runTurn` builds, in miniature: window AND not-yet-spent. */ + const report = (sessionId: string) => + store.noteCredentialRaceReport(sessionId) <= + CREDENTIAL_RACE_REPORTS_PER_SESSION; + + const classify = (sessionId: string) => + classifyRunError(new Error(ANTHROPIC_DIRECT_401), "claude", "anthropic", { + connection: { deployment: "direct" }, + daytonaCredentialFresh: () => report(sessionId), + }); + + it("tells the first refusal to retry and the second to add a key", () => { + // This is the test that would have caught the parked draft's hole: without the counter, a + // genuinely wrong key is told to retry forever, because the failed turn deletes the sandbox + // and the retry re-arms the freshness window. + assert.equal(classify("session-a").code, "credential_delivery_failed"); + + const second = classify("session-a"); + assert.equal(second.code, "runner_error"); + assert.match(second.message, /add the project's Anthropic key/); + }); + + it("stays on the add-a-key advice for every later refusal", () => { + classify("session-a"); + classify("session-a"); + assert.equal(classify("session-a").code, "runner_error"); + }); + + it("counts each session separately", () => { + assert.equal(classify("session-a").code, "credential_delivery_failed"); + assert.equal(classify("session-b").code, "credential_delivery_failed"); + assert.equal(classify("session-a").code, "runner_error"); + }); + + it("counts only when the classifier actually asks", () => { + // A turn that failed for an unrelated reason must not spend the session's one report. + classifyRunError(new Error("ETIMEDOUT"), "claude", "anthropic", { + daytonaCredentialFresh: () => report("session-c"), + }); + assert.equal(store.credentialRaceReportCount("session-c"), 0); + assert.equal(classify("session-c").code, "credential_delivery_failed"); + }); + + it("forgets the count when the session is cleared", () => { + classify("session-a"); + assert.equal(store.credentialRaceReportCount("session-a"), 1); + store.clear("session-a"); + assert.equal(store.credentialRaceReportCount("session-a"), 0); + assert.equal(classify("session-a").code, "credential_delivery_failed"); + }); +}); + +describe("the swallowed-Pi-error recovery path (Codex P1)", () => { + // Pi does not throw a provider refusal: it records it in its transcript and ends the turn + // cleanly, so the recovery path re-classifies it. That call site originally omitted the + // freshness predicate, which meant a credential race arriving THIS way was still reported as + // the user's key problem — the whole fix, bypassed by the harness that fails most quietly. + beforeEach(enableDaytonaProvider); + + const REMOTE_CWD = "/home/sandbox"; + + /** + * A model connection whose key rides a Daytona Secret — an `opaque_http` credential plus the + * exact-host endpoint the Secret is scoped to. Without BOTH the plan builds no model-secret + * candidate, nothing arms `modelSecretDeliveredAt`, and the test would pass against the very + * bug it exists to catch by never reaching the branch. + */ + const DAYTONA_MODEL_CONNECTION = { + provider: "anthropic", + deployment: "direct", + credentialMode: "env" as const, + endpoint: { baseUrl: "https://api.anthropic.com" }, + credentials: [ + { + binding: { kind: "environment" as const, name: "ANTHROPIC_API_KEY" }, + value: "sk-ant-fixture-value", + usage: "opaque_http" as const, + }, + ], + }; + + it("classifies a fresh-secret 401 recovered from the transcript as delivery", async () => { + const { result, events } = await runSilentTurn( + { + harness: "pi_core", + sandbox: "daytona", + modelConnection: DAYTONA_MODEL_CONNECTION, + }, + { + cwd: REMOTE_CWD, + sandboxTranscript: piTranscriptWithError( + REMOTE_CWD, + "API Error: 401 Invalid bearer token", + ), + }, + ); + + assert.equal(result.ok, false); + assert.equal(result.error, DELIVERY_MESSAGE); + // The playground renders the stream, not the envelope, so the honest copy has to be in both. + const errorEvent = events.find((event) => event.type === "error"); + assert.ok(errorEvent, "no error event in the stream"); + assert.equal(errorEvent.message, DELIVERY_MESSAGE); + }); + + it("still blames nothing on the user when the transcript refusal is unrelated", async () => { + // The guard in the other direction: the predicate must not turn every recovered failure into + // a delivery fault. + const { result } = await runSilentTurn( + { + harness: "pi_core", + sandbox: "daytona", + modelConnection: DAYTONA_MODEL_CONNECTION, + }, + { + cwd: REMOTE_CWD, + sandboxTranscript: piTranscriptWithError( + REMOTE_CWD, + "Rate limit reached for gpt-5 in organization org-abc on tokens per min.", + ), + }, + ); + + assert.equal(result.ok, false); + assert.equal( + result.error, + "Too many requests right now. Try again in a moment.", + ); + }); +}); + +describe("the fresh-Secret branch requires a provider 401 (CodeRabbit, #6408)", () => { + // `AUTH_REFUSAL` is broad on purpose — it decides which advice to print, and bare + // "unauthorized" appears in authorization failures all over the runner. That breadth is wrong + // for this branch, which SPENDS the session's one credential-race report and tells the user to + // retry. An unrelated "unauthorized" inside the propagation window would burn the report and + // hand out retry guidance a retry cannot fix, leaving the genuine race that followed to get the + // add-a-key copy. + const UNRELATED = [ + "Unauthorized: the tool's own API rejected the request", + "mount failed: unauthorized", + "authentication required by the storage backend", + "invalid api key for the analytics service", + ]; + + it("does not classify an unrelated authorization failure as a delivery race", () => { + for (const raw of UNRELATED) { + const r = classifyRunError(new Error(raw), "claude", "anthropic", { + daytonaCredentialFresh: () => true, + }); + assert.equal(r.code, "runner_error", raw); + assert.doesNotMatch( + r.message, + /credentials from reaching the model/, + raw, + ); + } + }); + + it("does not let an unrelated authorization failure consume the session report", () => { + const store = new SessionContinuityStore(); + const report = () => + store.noteCredentialRaceReport("session-x") <= + CREDENTIAL_RACE_REPORTS_PER_SESSION; + + classifyRunError(new Error(UNRELATED[0]), "claude", "anthropic", { + daytonaCredentialFresh: report, + }); + assert.equal(store.credentialRaceReportCount("session-x"), 0); + + // The report is still available for the real race that follows. + const r = classifyRunError( + new Error(ANTHROPIC_DIRECT_401), + "claude", + "anthropic", + { + daytonaCredentialFresh: report, + }, + ); + assert.equal(r.code, "credential_delivery_failed"); + }); + + it("still recognizes the 401 shapes providers actually send", () => { + for (const raw of [ + "API Error: 401 Invalid bearer token", + 'HTTP 401: {"error":"unauthorized"}', + '{"status_code": 401, "message": "no"}', + "http-401 refused", + ]) { + const r = classifyRunError(new Error(raw), "claude", "anthropic", { + daytonaCredentialFresh: () => true, + }); + assert.equal(r.code, "credential_delivery_failed", raw); + } + }); + + it("keeps the placeholder branches free of the report budget", () => { + // A body that echoes the placeholder is self-evidencing: a real user key never contains + // `dtn_`, so every such refusal IS a delivery failure however often it repeats. Capping it + // would eventually tell a user with a good key to add one. + const store = new SessionContinuityStore(); + const report = () => + store.noteCredentialRaceReport("session-y") <= + CREDENTIAL_RACE_REPORTS_PER_SESSION; + for (let i = 0; i < 5; i++) { + const r = classifyRunError( + new Error(LITELLM_PROXY_401), + "claude", + "anthropic", + { + daytonaCredentialFresh: report, + }, + ); + assert.equal(r.code, "credential_delivery_failed"); + } + assert.equal(store.credentialRaceReportCount("session-y"), 0); + }); +}); + +describe("a 401 the RUNNER produced is not the provider's (CodeRabbit Major, #6422)", () => { + // This classifier reads one flattened error STRING; it never sees an HTTP response, so the + // status it matches is whatever the throwing code wrote into the message. Several authenticated + // calls the runner makes DURING a turn can answer 401 and reach the same catch. Left + // unexcluded, any of them inside the propagation window spends the session's one report and + // prints retry guidance for a failure a retry cannot fix — and the genuine race that follows + // then gets the add-a-key copy, which is the original bug wearing a disguise. + // EXACTLY the five prefixed emitters that reach this classifier as input. Mount, geesefs and + // otel are deliberately absent: those sites build their message AROUND `conciseError`, so the + // prefix is added after classification and the classifier only ever sees the inner error. + const RUNNER_401 = [ + "tool call workflow.variant.summarizer failed: HTTP 401", + "attachment fetch failed: HTTP 401", + "attachment claim failed: HTTP 401", + "session records query failed: HTTP 401", + "session records persist failed: HTTP 401", + ]; + + it("does not classify a runner-side 401 as a credential race", () => { + for (const raw of RUNNER_401) { + const r = classifyRunError(new Error(raw), "claude", "anthropic", { + daytonaCredentialFresh: () => true, + }); + assert.equal(r.code, "runner_error", raw); + assert.doesNotMatch( + r.message, + /credentials from reaching the model/, + raw, + ); + } + }); + + it("does not let a runner-side 401 consume the session report", () => { + // The load-bearing half. A consumed report is invisible at the time and only shows up later, + // as the real race being told to add a key. + const store = new SessionContinuityStore(); + const report = () => + store.noteCredentialRaceReport("session-z") <= + CREDENTIAL_RACE_REPORTS_PER_SESSION; + + for (const raw of RUNNER_401) { + classifyRunError(new Error(raw), "claude", "anthropic", { + daytonaCredentialFresh: report, + }); + } + assert.equal(store.credentialRaceReportCount("session-z"), 0); + + // Still available for the genuine refusal that follows. + const r = classifyRunError( + new Error(ANTHROPIC_DIRECT_401), + "claude", + "anthropic", + { + daytonaCredentialFresh: report, + }, + ); + assert.equal(r.code, "credential_delivery_failed"); + }); + + it("does not exclude a provider 401 whose prose merely contains an emitter substring", () => { + // The exclusion must never fire on a PROVIDER refusal, because that is the worse direction of + // this bug: it hands a genuine race the add-a-key copy. An earlier draft matched loose + // `mount failed` and a bare `otel`, which swallowed all three of these. + for (const raw of [ + "API Error: 401 the requested amount failed to authorize", + "API Error: 401 paramount failed", + "API Error: 401 hotel-search rejected the key", + ]) { + const r = classifyRunError(new Error(raw), "claude", "anthropic", { + daytonaCredentialFresh: () => true, + }); + assert.equal(r.code, "credential_delivery_failed", raw); + } + }); + + it("still classifies a provider 401 that merely mentions a tool", () => { + // The exclusion keys on the runner's own failure PREFIX, not on any appearance of the word: + // a model refusal whose body happens to say "tool" must not be excluded. + const r = classifyRunError( + new Error( + 'API Error: 401 {"message":"Invalid bearer token","request":"tool_use"}', + ), + "claude", + "anthropic", + { daytonaCredentialFresh: () => true }, + ); + assert.equal(r.code, "credential_delivery_failed"); + }); + + it("keeps a self-evidencing placeholder echo classified inside a runner-side failure", () => { + // The placeholder branch runs earlier and stands alone: a literal `dtn_secret_` in the body + // means the placeholder really went out, whoever was calling, so the runner-side exclusion + // must not suppress it. + const r = classifyRunError( + new Error("tool call x failed: HTTP 401 rejected dtn_secret_abc123"), + "claude", + "anthropic", + { daytonaCredentialFresh: () => true }, + ); + assert.equal(r.code, "credential_delivery_failed"); + }); + + it("refuses a bare `dtn_****` mask with no LiteLLM phrasing", () => { + // Not a regression: `Received=dtn_****` is evidence only via LiteLLM's quoted sentence. On + // its own the mask has a zero-length stem, which the tightened signature rejects by design so + // a literal glob cannot spoof it. Pinned here so the asymmetry is deliberate, not accidental. + const r = classifyRunError( + new Error("tool call x failed: HTTP 401 got dtn_**** instead"), + "claude", + "anthropic", + { daytonaCredentialFresh: () => true }, + ); + assert.equal(r.code, "runner_error"); + + // With LiteLLM's own sentence in front of it, the same mask IS evidence. + const withPhrase = classifyRunError( + new Error( + "LiteLLM Virtual Key expected. Received=dtn_****, expected sk-", + ), + "claude", + "anthropic", + { daytonaCredentialFresh: () => false }, + ); + assert.equal(withPhrase.code, "credential_delivery_failed"); + }); +}); diff --git a/services/runner/tests/unit/daytona-secret-provider.test.ts b/services/runner/tests/unit/daytona-secret-provider.test.ts index 6268f933f5..0126461000 100644 --- a/services/runner/tests/unit/daytona-secret-provider.test.ts +++ b/services/runner/tests/unit/daytona-secret-provider.test.ts @@ -252,7 +252,9 @@ describe("process-local Daytona Secret provider", () => { false, ); assert.equal(events.includes("sandbox:destroy"), false); - assert.match(logs[0], /retaining 2 Secret allocation/); + // logs[0] is now the allocation timing line; the retention notice follows it. + assert.match(logs[0], /\[daytona-secrets\] allocated n=2/); + assert.match(logs[1], /retaining 2 Secret allocation/); }); it("deletes Secrets when provider construction proves no remote create started", async () => { diff --git a/services/runner/tests/unit/harness-kind.test.ts b/services/runner/tests/unit/harness-kind.test.ts new file mode 100644 index 0000000000..e8698b3cd3 --- /dev/null +++ b/services/runner/tests/unit/harness-kind.test.ts @@ -0,0 +1,72 @@ +/** + * The one harness-identity normalizer (audit findings 3 and 6). + * + * Run: pnpm exec vitest run tests/unit/harness-kind.test.ts + */ +import { describe, it } from "vitest"; +import assert from "node:assert/strict"; + +import { + harnessKindOf, + normalizedHarnessMode, +} from "../../src/harness-kind.ts"; +import { harnessKind } from "../../src/lifecycle/reconciliation-router.ts"; +import type { AgentRunRequest } from "../../src/protocol.ts"; + +describe("harnessKindOf", () => { + it("round-trips every wire spelling the SDK can send", () => { + // The SDK's HarnessKind enum plus the legacy spelling and the empty default. A value + // added to the SDK without a row here must fail THIS test, not fall into `unknown` in + // production the way "pi_core" once did (#6364). + const table: Array<[string | undefined, string]> = [ + ["pi_core", "pi"], + ["pi_agenta", "pi"], // removed experiment; old stored configs still carry it + ["pi", "pi"], // never on the wire, accepted defensively + ["claude", "claude"], + ["codex", "codex"], + ["", "pi"], // empty defaults to pi_core + [undefined, "pi"], + ["future-thing", "unknown"], // fail closed + // An unchecked `JSON.parse` can put a non-string here; each must fail closed rather + // than borrow Pi's default through a falsy `||` (#6364's review guard). + [null as never, "unknown"], + [0 as never, "unknown"], + [false as never, "unknown"], + ]; + for (const [wire, kind] of table) { + assert.equal(harnessKindOf(wire), kind, `harnessKindOf(${wire})`); + } + }); + + it("is the same answer the lifecycle router gives", () => { + for (const harness of ["pi_core", "pi_agenta", "claude", "codex", "x"]) { + assert.equal( + harnessKind({ harness } as AgentRunRequest), + harnessKindOf(harness), + harness, + ); + } + }); +}); + +describe("normalizedHarnessMode", () => { + it("resolves only for codex, where the default and an absent value are equal", () => { + assert.equal( + normalizedHarnessMode("codex", undefined), + "agent-full-access", + ); + assert.equal( + normalizedHarnessMode("codex", "agent-full-access"), + "agent-full-access", + "an explicitly-sent default equals an absent field", + ); + assert.equal(normalizedHarnessMode("codex", "read-only"), "read-only"); + assert.equal( + normalizedHarnessMode("codex", "not-a-mode"), + "agent-full-access", + ); + // Every non-codex harness ignores the field entirely. + assert.equal(normalizedHarnessMode("pi_core", "read-only"), null); + assert.equal(normalizedHarnessMode("claude", "read-only"), null); + }); +}); diff --git a/services/runner/tests/unit/lifecycle-apply-plan.test.ts b/services/runner/tests/unit/lifecycle-apply-plan.test.ts index a2f02f0fa6..99e098526e 100644 --- a/services/runner/tests/unit/lifecycle-apply-plan.test.ts +++ b/services/runner/tests/unit/lifecycle-apply-plan.test.ts @@ -217,4 +217,33 @@ describe("applyReconcilePlan: refresh-workspace installs the INCOMING configurat assert.equal(applied, false); assert.equal(committed.length, 0); }); + + it("refuses an apply-live action for a facet it cannot install (audit finding 7)", async () => { + // The arm used to treat EVERY `apply-live` as a model change. The day another facet routes + // here (the credential plan is the expected first), that would install the wrong thing and + // commit the new configuration. It must fail into a rebuild instead, without calling the + // model applier at all. + const env = makeEnv(); + const request: AgentRunRequest = { + harness: "claude", + model: "m1", + messages: [], + } as never; + const plan = buildPlan( + [{ facet: "runtime", kind: "apply-live", reason: "r" }], + ["runtime"], + ); + + let modelApplierCalled = false; + const applied = await applyReconcilePlan(env, request, plan, () => {}, { + applyModel: async () => { + modelApplierCalled = true; + return "m1"; + }, + }); + + assert.equal(applied, false, "the caller must rebuild"); + assert.equal(committed.length, 0, "and applied state must not advance"); + assert.equal(modelApplierCalled, false, "the model applier must not run"); + }); }); diff --git a/services/runner/tests/unit/lifecycle-desired-state.test.ts b/services/runner/tests/unit/lifecycle-desired-state.test.ts index e83dd692a1..625d83b425 100644 --- a/services/runner/tests/unit/lifecycle-desired-state.test.ts +++ b/services/runner/tests/unit/lifecycle-desired-state.test.ts @@ -45,8 +45,16 @@ describe("facet ownership: one field moves exactly one facet", () => { overrides: Partial; facet: Facet; }> = [ - { what: "the sandbox provider", overrides: { sandbox: "daytona" }, facet: "sandbox" }, - { what: "the harness kind", overrides: { harness: "pi" }, facet: "sandbox" }, + { + what: "the sandbox provider", + overrides: { sandbox: "daytona" }, + facet: "sandbox", + }, + { + what: "the harness kind", + overrides: { harness: "pi" }, + facet: "sandbox", + }, { what: "the sandbox permission", overrides: { sandboxPermission: "none" as never }, @@ -69,7 +77,11 @@ describe("facet ownership: one field moves exactly one facet", () => { overrides: { agentsMd: "new instructions" }, facet: "workspaceFiles", }, - { what: "the system prompt", overrides: { systemPrompt: "sp" }, facet: "prompts" }, + { + what: "the system prompt", + overrides: { systemPrompt: "sp" }, + facet: "prompts", + }, { what: "the skills", overrides: { @@ -98,13 +110,9 @@ describe("facet ownership: one field moves exactly one facet", () => { overrides: { customTools: [{ name: "t" }] as never }, facet: "toolCatalog", }, - { - what: "the tool callback endpoint", - overrides: { - toolCallback: { endpoint: "https://gateway/tools/call" } as never, - }, - facet: "toolCatalog", - }, + // The tool callback endpoint left this table with audit finding 5: it is read from the + // incoming request every turn, so it moves NO facet. The per-turn-volatile suite below + // pins that instead. ]; for (const { what, overrides, facet } of cases) { @@ -145,11 +153,89 @@ describe("facet normalization: stability and coverage", () => { { messages: [{ role: "user" as const, content: "different" }] }, { turnId: "another-turn" }, { context: { propagation: { traceparent: "00-abc-def-01" } } as never }, + // The resolved model's input modalities ride the request per turn and change with the + // model; hashing them refused the live route on any cross-modality model switch. + { modelCapabilities: { inputModalities: ["text"] } as never }, + // The rest of runContext is per-turn metadata: a committed revision or a trace id must + // never evict. Only `workflow.artifact.id` is identity (it selects the agent mount). + { + runContext: { + workflow: { + revision: { id: "rev-2", version: "7" }, + variant: { id: "var-2" }, + }, + trace: { trace_id: "abc" }, + } as never, + }, + // The per-deployment gateway URL is read from the incoming request every turn + // (finding 5); a moved deployment must not evict every warm session. + { toolCallback: { endpoint: "https://gateway-2/tools/call" } as never }, ]) { - assert.deepEqual(movedBy(overrides), [], JSON.stringify(overrides).slice(0, 40)); + const label = JSON.stringify(overrides).slice(0, 40); + assert.deepEqual(movedBy(overrides), [], label); + // BOTH identity views must ignore a volatile: a field that sneaks back into the + // fingerprint alone would cold-evict every warm session while this facet probe + // stayed green (Codex review of the finding-5 change). + assert.equal( + configFingerprint({ ...BASE, ...overrides } as AgentRunRequest), + configFingerprint(BASE), + `fingerprint moved: ${label}`, + ); } }); + it("omitted MCP credentials equal an empty credential array, in BOTH views", () => { + // The facet digest normalized an omitted array to [] while the fingerprint kept the + // omission, so two identical requests disagreed in one view only: a cold evict with an + // empty live plan and a DISAGREE log (Codex review of the finding-3 change). + const server = { name: "s", connection: { url: "https://mcp.test" } }; + const omitted = { ...BASE, mcpServers: [server] } as never as AgentRunRequest; + const empty = { + ...BASE, + mcpServers: [{ ...server, connection: { ...server.connection, credentials: [] } }], + } as never as AgentRunRequest; + assert.equal(configFingerprint(omitted), configFingerprint(empty)); + assert.deepEqual(digestsOf(omitted), digestsOf(empty)); + }); + + it("the fingerprint and the facets agree about harness-mode changes (finding 3)", () => { + // The fingerprint normalized the Codex mode while the facet took it raw, so an + // explicitly-sent default moved `harnessSession` but not the fingerprint — and a session + // poisoned that way rebuilt on every later mixed plan. Both views now share one + // normalizer; this pins the agreement in BOTH directions. + const codex = { ...BASE, harness: "codex" } as AgentRunRequest; + const agree = (a: AgentRunRequest, b: AgentRunRequest, why: string) => { + const fpMoved = configFingerprint(a) !== configFingerprint(b); + const facetsMoved = + JSON.stringify(digestsOf(a)) !== JSON.stringify(digestsOf(b)); + assert.equal(fpMoved, facetsMoved, why); + return fpMoved; + }; + assert.equal( + agree( + codex, + { ...codex, harnessMode: "agent-full-access" }, + "explicit default", + ), + false, + "an explicitly-sent default equals an absent field in both views", + ); + assert.equal( + agree(codex, { ...codex, harnessMode: "read-only" }, "real mode change"), + true, + "a real Codex mode change moves both views", + ); + assert.equal( + agree( + BASE, + { ...BASE, harnessMode: "read-only" } as AgentRunRequest, + "non-codex", + ), + false, + "a mode on a harness that ignores it moves neither view", + ); + }); + it("NO INPUT DRIFT: a field that moves the fingerprint also moves a facet", () => { // The load-bearing invariant. The shadow comparison is only meaningful when the facets see // exactly what the fingerprint sees. A field in the fingerprint but in no facet would make @@ -157,25 +243,32 @@ describe("facet normalization: stability and coverage", () => { const probes: Array<[string, Partial]> = [ ["sandbox", { sandbox: "daytona" }], ["harness", { harness: "pi" }], + [ + "runContext.workflow.artifact.id", + { runContext: { workflow: { artifact: { id: "art-2" } } } } as never, + ], ["model", { model: "m2" }], ["agentsMd", { agentsMd: "x" }], ["systemPrompt", { systemPrompt: "x" }], ["appendSystemPrompt", { appendSystemPrompt: "x" }], - ["skills", { skills: [{ name: "s", description: "d", body: "b" }] as never }], + [ + "skills", + { skills: [{ name: "s", description: "d", body: "b" }] as never }, + ], ["customTools", { customTools: [{ name: "t" }] as never }], - ["harnessFiles", { harnessFiles: [{ path: "a", content: "b" }] as never }], + [ + "harnessFiles", + { harnessFiles: [{ path: "a", content: "b" }] as never }, + ], ["permissions", { permissions: { default: "deny" } as never }], ["sandboxPermission", { sandboxPermission: "none" as never }], ["mcpServers", { mcpServers: [{ name: "x", connection: {} }] as never }], - ["modelCapabilities", { modelCapabilities: { vision: true } as never }], - [ - "toolCallback.endpoint", - { toolCallback: { endpoint: "https://gateway/tools/call" } as never }, - ], ]; for (const [name, overrides] of probes) { - const changed = configFingerprint({ ...BASE, ...overrides }) !== configFingerprint(BASE); + const changed = + configFingerprint({ ...BASE, ...overrides }) !== + configFingerprint(BASE); assert.ok(changed, `precondition: ${name} must move the fingerprint`); assert.notDeepEqual( movedBy(overrides), @@ -205,7 +298,10 @@ describe("facet normalization: stability and coverage", () => { } as never, }); assert.deepEqual( - changedFacets(digestsOf(withSecret("sk-a")), digestsOf(withSecret("sk-b"))), + changedFacets( + digestsOf(withSecret("sk-a")), + digestsOf(withSecret("sk-b")), + ), [], "only the credential SHAPE is hashed, never its value", ); @@ -235,9 +331,8 @@ describe("facet normalization: stability and coverage", () => { // Same reasoning on the session side. Section 1.4 exempts permission TIGHTENING from // apply-live entirely, so a permissions change must not ride the `setModel` route. assert.deepEqual(movedBy({ model: "m2" }), ["model"]); - assert.deepEqual( - movedBy({ permissions: { default: "deny" } as never }), - ["harnessSession"], - ); + assert.deepEqual(movedBy({ permissions: { default: "deny" } as never }), [ + "harnessSession", + ]); }); }); diff --git a/services/runner/tests/unit/lifecycle-live-routes.test.ts b/services/runner/tests/unit/lifecycle-live-routes.test.ts index 105f4df894..316d50bfd3 100644 --- a/services/runner/tests/unit/lifecycle-live-routes.test.ts +++ b/services/runner/tests/unit/lifecycle-live-routes.test.ts @@ -167,6 +167,7 @@ function makeEngine(options: EngineOptions = {}) { (env as unknown as FakeEnv).commitApplied({ configFingerprint: fp, facets: normalizeDesiredState(request, fp).digests, + fieldDigests: {}, }); return true; }, @@ -412,7 +413,10 @@ describe("THE RELEASE BLOCKER: an instructions change REBUILDS", () => { await runWithKeepalive(next, undefined, undefined, ctx); assert.equal(env1.appliedState.configFingerprint, fingerprintBefore); - assert.notEqual(env1.appliedState.configFingerprint, configFingerprint(next)); + assert.notEqual( + env1.appliedState.configFingerprint, + configFingerprint(next), + ); assert.equal(env1.appliedState.generation, before); }); @@ -687,6 +691,149 @@ describe("FAIL CLOSED: everything that must still rebuild", () => { }); }); +describe("A REPAIR ANSWERS ONLY ITS OWN QUESTION (audit finding 1)", () => { + // The live route used to set `mismatch = undefined` wholesale, so a model switch riding with + // an edited transcript, a rotated credential, or a stale tail cleared THOSE checks too and + // continued warm on an environment that failed them. Each case here pairs the live-applicable + // model change with one other mismatch and asserts the rebuild still happens. The wasted + // in-place apply before the rebuild is acceptable; the reuse was not. + + it("a model change plus an EDITED transcript still rebuilds", async () => { + const { engine, calls } = makeEngine(); + const ctx = makeCtx(engine); + await runWithKeepalive(turn1, undefined, undefined, ctx); + await runWithKeepalive( + turn2({ + model: "m2", + messages: [ + { role: "user", content: "EDITED" }, + { role: "assistant", content: "hi" }, + { role: "user", content: "more" }, + ], + }), + undefined, + undefined, + ctx, + ); + assert.equal( + calls.acquire, + 2, + "the edited history must evict, model route or not", + ); + }); + + it("a model change plus a ROTATED credential with no delivery port still rebuilds", async () => { + const { engine, calls } = makeEngine(); + const ctx = makeCtx(engine); + await runWithKeepalive( + withSecret("sk-a", turn1), + undefined, + undefined, + ctx, + ); + await runWithKeepalive( + withSecret("sk-b", turn2({ model: "m2" })), + undefined, + undefined, + ctx, + ); + assert.equal( + calls.acquire, + 2, + "an undeliverable rotation must evict; the old key must never serve the turn", + ); + }); + + it("a model change plus a DELIVERABLE rotation chains both repairs and stays warm", async () => { + const { port, deliveries } = makeCredentialPort(); + const { engine, calls } = makeEngine({ credentialPort: port }); + const ctx = makeCtx(engine); + await runWithKeepalive( + withSecret("sk-a", turn1), + undefined, + undefined, + ctx, + ); + await runWithKeepalive( + withSecret("sk-b", turn2({ model: "m2" })), + undefined, + undefined, + ctx, + ); + assert.equal( + calls.acquire, + 1, + "both doors repaired their own reason: warm", + ); + assert.equal(deliveries.length, 1, "the rotation was actually delivered"); + assert.equal( + calls.applied.length, + 1, + "the model change was actually applied", + ); + }); + + it("a model change plus an edited transcript plus an undeliverable rotation DELETES", async () => { + // The eviction is named by the FIRST unresolved reason and disposed by ALL of them. + // `history` sorts ahead of the credential checks, so this combination was evicted as + // `history`, mapped to `continuity-invalid`, and PARKED — handing the next turn a sandbox + // whose daemon still held the old key. The name may stay `history`; the disposition may not. + const { engine, calls } = makeEngine(); + const ctx = makeCtx(engine); + await runWithKeepalive( + withSecret("sk-a", turn1), + undefined, + undefined, + ctx, + ); + await runWithKeepalive( + withSecret( + "sk-b", + turn2({ + model: "m2", + messages: [ + { role: "user", content: "EDITED" }, + { role: "assistant", content: "hi" }, + { role: "user", content: "more" }, + ], + }), + ), + undefined, + undefined, + ctx, + ); + assert.equal(calls.acquire, 2, "the combination must evict"); + assert.deepEqual( + calls.acquiredEnvs[0]?.destroyReasons, + ["runtime-incompatible"], + "a stale credential outranks a continuity-only park", + ); + }); + + it("a model change plus a STALE tail still rebuilds", async () => { + const { engine, calls } = makeEngine(); + const ctx = makeCtx(engine); + await runWithKeepalive(turn1, undefined, undefined, ctx); + await runWithKeepalive( + turn2({ + model: "m2", + messages: [ + { role: "user", content: "hello" }, + { role: "assistant", content: "hi" }, + ], + }), + undefined, + undefined, + ctx, + ); + assert.equal( + calls.acquire, + 2, + "a tail that is not a fresh user turn must evict", + ); + }); +}); + describe("the disagreement counters stay quiet", () => { it("an instructions change agrees on a REBUILD, so the withdrawal is silent too", async () => { // Why the fix moved the capability table and not only the live set. Dropping diff --git a/services/runner/tests/unit/lifecycle-reconcile-plan.test.ts b/services/runner/tests/unit/lifecycle-reconcile-plan.test.ts index c68f9caecc..0fdef4b20c 100644 --- a/services/runner/tests/unit/lifecycle-reconcile-plan.test.ts +++ b/services/runner/tests/unit/lifecycle-reconcile-plan.test.ts @@ -246,6 +246,20 @@ describe("plan construction per facet", () => { it("an unknown harness fails closed to a rebuild", () => { assert.equal(harnessKind({ ...BASE, harness: "future-thing" } as never), "unknown"); + // The wire spellings of Pi, and the empty default, all resolve to the "pi" capability row — + // "pi_core" landing in `unknown` sent every playground run to the fail-closed all-rebuild row. + assert.equal(harnessKind({ ...BASE, harness: "pi_core" } as never), "pi"); + assert.equal(harnessKind({ ...BASE, harness: "pi_agenta" } as never), "pi"); + assert.equal(harnessKind({ ...BASE, harness: undefined } as never), "pi"); + // `/stream` decodes its body unchecked, so a non-string can reach here. It must not + // borrow Pi's live routes through the `|| "pi_core"` default. + for (const junk of [null, 0, false, 1, {}, []]) { + assert.equal(harnessKind({ ...BASE, harness: junk } as never), "unknown"); + } + assert.equal( + capabilitiesFor({ ...BASE, harness: "pi_core" } as never).model, + "apply-live", + ); const capabilities = capabilitiesFor({ ...BASE, harness: "future-thing" } as never); assert.equal(capabilities.toolCatalog, "rebuild-sandbox"); assert.equal(capabilities.workspaceFiles, "rebuild-sandbox"); diff --git a/services/runner/tests/unit/platform-credential-attribution.test.ts b/services/runner/tests/unit/platform-credential-attribution.test.ts index 4c30e2f872..71d44b3183 100644 --- a/services/runner/tests/unit/platform-credential-attribution.test.ts +++ b/services/runner/tests/unit/platform-credential-attribution.test.ts @@ -178,6 +178,195 @@ describe("platform credential attribution", () => { }); }); +/** + * F1: a self-hoster sets `AGENTA_API_URL` to a localhost URL, which is the natural value for a + * local docker-compose stack. The api rewrites that host to `host.docker.internal` before it + * dispatches the run, so the endpoint the runner is handed can never string-match the raw env + * value it holds, and the strict branch drops the credential on a correctly configured platform. + * Repro: probe session f9483f76 on the rel1144 stack, "trace endpoint host host.docker.internal:8480 + * is not Agenta ingest ... dropping the run credential". + */ +describe("bridge-rewritten localhost ingest", () => { + const LOCAL_BASE = "http://localhost:8480/api"; + const BRIDGE_ENDPOINT = "http://host.docker.internal:8480/api/otlp/v1/traces"; + + it("uses the credential when a localhost base is handed its bridge-rewritten endpoint", () => { + vi.stubEnv("AGENTA_API_URL", LOCAL_BASE); + const { lines, log } = withLog(); + + assert.equal( + platformCredentialForRequest(request(BRIDGE_ENDPOINT), log), + CREDENTIAL, + ); + assert.deepEqual(lines, []); + }); + + it("uses the credential for the 0.0.0.0 form the api rewrites the same way", () => { + vi.stubEnv("AGENTA_API_URL", "http://0.0.0.0:8480/api"); + const { lines, log } = withLog(); + + assert.equal( + platformCredentialForRequest(request(BRIDGE_ENDPOINT), log), + CREDENTIAL, + ); + assert.deepEqual(lines, []); + }); + + it("admits the bridge form of the internal hop too", () => { + vi.stubEnv("AGENTA_API_INTERNAL_URL", "http://localhost:8000"); + vi.stubEnv("AGENTA_API_URL", PUBLIC_BASE); + const { lines, log } = withLog(); + + assert.equal( + platformCredentialForRequest( + request("http://host.docker.internal:8000/otlp/v1/traces"), + log, + ), + CREDENTIAL, + ); + assert.deepEqual(lines, []); + }); + + it("still admits the localhost endpoint itself, for a host-network deployment", () => { + // `parse_url` returns the url unchanged when the api runs with network mode "host", so the + // raw match this fix widens must keep working exactly as before. + vi.stubEnv("AGENTA_API_URL", LOCAL_BASE); + const { lines, log } = withLog(); + + assert.equal( + platformCredentialForRequest( + request("http://localhost:8480/api/otlp/v1/traces"), + log, + ), + CREDENTIAL, + ); + assert.deepEqual(lines, []); + }); + + it("does not admit host.docker.internal on another port", () => { + // The mirror carries the port across, so a different port is a different deployment. + vi.stubEnv("AGENTA_API_URL", LOCAL_BASE); + const { lines, log } = withLog(); + + assert.equal( + platformCredentialForRequest( + request("http://host.docker.internal:9999/api/otlp/v1/traces"), + log, + ), + "", + ); + assert.equal(lines.length, 1); + assert.match(lines[0]!, /dropping the run credential/); + }); + + it("does not admit host.docker.internal on another path", () => { + vi.stubEnv("AGENTA_API_URL", LOCAL_BASE); + const { lines, log } = withLog(); + + assert.equal( + platformCredentialForRequest( + request("http://host.docker.internal:8480/collector/v1/traces"), + log, + ), + "", + ); + assert.equal(lines.length, 1); + assert.match(lines[0]!, /dropping the run credential/); + }); + + it("keeps dropping an unrelated host when the base is a localhost url", () => { + vi.stubEnv("AGENTA_API_URL", LOCAL_BASE); + const { lines, log } = withLog(); + + assert.equal( + platformCredentialForRequest( + request("https://collector.thirdparty.example/v1/traces"), + log, + ), + "", + ); + assert.equal(lines.length, 1); + assert.match(lines[0]!, /dropping the run credential/); + assert.match(lines[0]!, /collector\.thirdparty\.example/); + }); + + it("leaves a 127.0.0.1 base alone, because the api does not rewrite that host", () => { + // `parse_url` rewrites only `localhost` and `0.0.0.0`, so a 127.0.0.1 deployment matches + // itself and never sees the bridge form. Admitting it would widen the allowlist past the + // platform's own rewrite. + vi.stubEnv("AGENTA_API_URL", "http://127.0.0.1:8480/api"); + const { lines, log } = withLog(); + + assert.equal( + platformCredentialForRequest( + request("http://127.0.0.1:8480/api/otlp/v1/traces"), + log, + ), + CREDENTIAL, + ); + assert.deepEqual(lines, []); + + resetPlatformCredentialWarnings(); + assert.equal( + platformCredentialForRequest(request(BRIDGE_ENDPOINT), log), + "", + ); + assert.equal(lines.length, 1); + assert.match(lines[0]!, /dropping the run credential/); + }); + + it("still admits the localhost endpoint host mode dispatches, with no alias involved", () => { + // The SDK's `parse_url` rewrites only when DOCKER_NETWORK_MODE is exactly "bridge". In + // `host` mode, and when the var is unset, the platform dispatches the localhost endpoint + // unchanged, so the raw base is what has to match and the alias is simply unused. The + // runner cannot see the mode (no env_file by design, and the var is not in its compose + // `environment:` block), and it cannot tell "unset" from "bridge but invisible", so the + // alias is emitted unconditionally rather than gated. See `bridgeRewrittenBase`. + vi.stubEnv("AGENTA_API_URL", LOCAL_BASE); + const { lines, log } = withLog(); + + assert.equal( + platformCredentialForRequest( + request("http://localhost:8480/api/otlp/v1/traces"), + log, + ), + CREDENTIAL, + ); + assert.deepEqual(lines, []); + }); + + it("derives no alias from a scheme-less base, which no aliasing could rescue", () => { + // The SDK's `parse_url` does no scheme defaulting, so a scheme-less `AGENTA_API_URL` + // produces an equally scheme-less ENDPOINT: `new URL` reads "localhost:" as the scheme and + // both sides normalize to nonsense. Admitting the bridge form here would fix nothing and + // would mirror the api's copy of `parse_url`, which is not what shapes this value. Such a + // deployment is broken upstream, at the exporter target itself. + vi.stubEnv("AGENTA_API_URL", "localhost:8480/api"); + const { lines, log } = withLog(); + + assert.equal( + platformCredentialForRequest(request(BRIDGE_ENDPOINT), log), + "", + ); + assert.equal(lines.length, 1); + assert.match(lines[0]!, /dropping the run credential/); + }); + + it("does not arm the strict branch when only a localhost internal hop is configured", () => { + // The unconfigured-base behaviour must stay exactly as it is: the runner keeps the credential + // and names the gap rather than failing closed on an undecidable attribution. + vi.stubEnv("AGENTA_API_INTERNAL_URL", "http://localhost:8000"); + const { lines, log } = withLog(); + + assert.equal( + platformCredentialForRequest(request(PUBLIC_ENDPOINT), log), + CREDENTIAL, + ); + assert.equal(lines.length, 1); + assert.match(lines[0]!, /AGENTA_API_URL is not set/); + }); +}); + describe("publicApiBaseConfigured", () => { it("is false when only the internal hop is set", () => { vi.stubEnv("AGENTA_API_INTERNAL_URL", INTERNAL_BASE); diff --git a/services/runner/tests/unit/sandbox-agent-errors.test.ts b/services/runner/tests/unit/sandbox-agent-errors.test.ts index 975644c10b..d325c4c8cf 100644 --- a/services/runner/tests/unit/sandbox-agent-errors.test.ts +++ b/services/runner/tests/unit/sandbox-agent-errors.test.ts @@ -92,6 +92,62 @@ describe("conciseError", () => { ); }); + it("classifies an unsubstituted Daytona placeholder as credential delivery, not the user's key", () => { + // The LiteLLM refusal body when the sandbox's opaque placeholder reaches it raw: the user's + // key is fine, so the add-a-key advice would be wrong (found live, 2026-08-29, EU cloud). + const result = classifyRunError( + new Error( + "401 LiteLLM Virtual Key expected. Received=dtn_****9maz, expected to start with 'sk-'.", + ), + "pi_core", + "openai", + ); + assert.equal(result.code, "credential_delivery_failed"); + assert.equal( + result.message, + "A temporary issue kept this run's credentials from reaching the model. Send the message again.", + ); + }); + + it("classifies a raw dtn_secret_ placeholder echo the same way", () => { + const result = classifyRunError( + new Error("401 Unauthorized: invalid api key 'dtn_secret_abc123'"), + "pi_core", + "openai", + ); + assert.equal(result.code, "credential_delivery_failed"); + }); + + it("names the connection neutrally, not the dialect family, for a custom-deployment auth failure", () => { + // A custom OpenAI-compatible connection resolves provider family "openai" for its DIALECT. + // A Gemini run through such a connection must not read "add the project's OpenAI key". + // And the hint must not name the SLUG: the runner cannot tell a user-created connection + // from a managed hidden one (starter-credits), so a slug can be an internal identifier + // pointing at a connection the user cannot edit (review finding on #6362). + const line = conciseError( + new Error("Authentication required"), + "pi_core", + "openai", + { + connection: { slug: "starter-credits", deployment: "custom" }, + }, + ); + assert.equal( + line, + "pi_core: model authentication failed — add the model connection's API key to the project vault, or log in (OAuth).", + ); + assert.doesNotMatch(line, /starter-credits/); + }); + + it("keeps the family hint when the deployment is not custom", () => { + assert.equal( + conciseError(new Error("Authentication required"), "pi_core", "openai", { + connection: { slug: "openai", deployment: "direct" }, + }), + "pi_core: model authentication failed — add the project's OpenAI key to the project vault, or log in (OAuth).", + ); + }); + it("formats a corrupt image provider error as a friendly message", () => { assert.equal( conciseError( diff --git a/services/runner/tests/unit/sandbox-agent-run-plan.test.ts b/services/runner/tests/unit/sandbox-agent-run-plan.test.ts index f32bd00af7..7ec08b13a8 100644 --- a/services/runner/tests/unit/sandbox-agent-run-plan.test.ts +++ b/services/runner/tests/unit/sandbox-agent-run-plan.test.ts @@ -59,6 +59,20 @@ describe("buildRunPlan", () => { assert.equal(created, false); }); + it("refuses a non-string harness instead of running it as Pi", () => { + // `/stream` decodes with an unchecked JSON.parse, so `null`/`0`/`false` can land here. The + // lifecycle router classifies those `unknown` (fail closed); the plan must not quietly + // default them to Pi and diverge (Codex review of the harness-normalizer change). + for (const junk of [null, 0, false, {}]) { + const result = buildRunPlan({ + harness: junk, + messages: [{ role: "user", content: "hi" }], + } as never); + assert.equal(result.ok, false, JSON.stringify(junk)); + if (!result.ok) assert.match(result.error, /harness/i); + } + }); + it("accepts an attachment-only current user turn", () => { const result = buildRunPlan( { @@ -357,10 +371,16 @@ describe("buildRunPlan", () => { assert.equal(result.plan.sandboxId, "local"); assert.equal(result.plan.workspace.cwd, "/tmp/local-cwd"); // Relay and telemetry are ephemeral runner files kept OFF the (possibly geesefs) cwd. - assert.ok(!result.plan.workspace.relayDir.startsWith(result.plan.workspace.cwd)); - assert.ok(result.plan.workspace.relayDir.endsWith("/agenta/relay/local-cwd")); assert.ok( - result.plan.workspace.telemetryDir.endsWith("/agenta/telemetry/local-cwd"), + !result.plan.workspace.relayDir.startsWith(result.plan.workspace.cwd), + ); + assert.ok( + result.plan.workspace.relayDir.endsWith("/agenta/relay/local-cwd"), + ); + assert.ok( + result.plan.workspace.telemetryDir.endsWith( + "/agenta/telemetry/local-cwd", + ), ); assert.equal( result.plan.workspace.usageOutPath, @@ -372,7 +392,9 @@ describe("buildRunPlan", () => { assert.equal(result.plan.prompt.appendSystemPrompt, "append"); assert.equal(result.plan.prompt.hasSystemPrompt, true); assert.equal(result.plan.credentials.hasApiKey, true); - assert.deepEqual(result.plan.credentials.modelEnvironment, { OPENAI_API_KEY: "key" }); + assert.deepEqual(result.plan.credentials.modelEnvironment, { + OPENAI_API_KEY: "key", + }); assert.equal(result.plan.workspace.sourcePiAgentDir, "/tmp/pi-agent"); assert.deepEqual( result.plan.tools.executableToolSpecs.map((tool) => tool.name), @@ -762,9 +784,14 @@ describe("buildRunPlan", () => { result.plan.workspace.toolMcpDir, "/home/sandbox/agenta/tool-mcp/agenta-fixed", ); - assert.notEqual(result.plan.workspace.toolMcpDir, result.plan.workspace.relayDir); + assert.notEqual( + result.plan.workspace.toolMcpDir, + result.plan.workspace.relayDir, + ); assert.ok( - !result.plan.workspace.toolMcpDir.startsWith(`${result.plan.workspace.relayDir}/`), + !result.plan.workspace.toolMcpDir.startsWith( + `${result.plan.workspace.relayDir}/`, + ), "the shim dir is never nested inside the relay dir (the relay loop sweeps it)", ); assert.equal( @@ -896,7 +923,10 @@ describe("buildRunPlan", () => { result.plan.tools.executableToolSpecs.map((tool) => tool.name), ["server_tool"], ); - assert.equal(result.plan.tools.clientToolPauseDisposition, "cold-acknowledge"); + assert.equal( + result.plan.tools.clientToolPauseDisposition, + "cold-acknowledge", + ); }); it("allows claude x daytona x client-ONLY tools (the shim advertises them and the relay parks)", () => { @@ -1184,8 +1214,14 @@ describe("buildRunPlan", () => { // The FULL materialized environment sets hasApiKey: on a Daytona Secrets run the opaque key // leaves the plaintext env for the secret plan, but the harness still receives its binding. assert.equal(result.plan.credentials.hasApiKey, true); - assert.equal(result.plan.credentials.modelEnvironment.ANTHROPIC_API_KEY, undefined); - assert.equal(result.plan.credentials.daytonaSecretPlan?.candidates.length, 1); + assert.equal( + result.plan.credentials.modelEnvironment.ANTHROPIC_API_KEY, + undefined, + ); + assert.equal( + result.plan.credentials.daytonaSecretPlan?.candidates.length, + 1, + ); // The resolved credentialMode is carried onto the plan (drives clear-then-apply). assert.equal(result.plan.credentials.credentialMode, "env"); assert.equal(result.plan.prompt.systemPrompt, undefined); @@ -1226,8 +1262,14 @@ describe("buildRunPlan", () => { const flagOn = buildRunPlan(localUseRequest, deps); assert.equal(flagOn.ok, true); if (!flagOn.ok) return; - assert.ok(flagOn.plan.credentials.daytonaSecretPlan, "flag on keeps the empty plan"); - assert.equal(flagOn.plan.credentials.daytonaSecretPlan.candidates.length, 0); + assert.ok( + flagOn.plan.credentials.daytonaSecretPlan, + "flag on keeps the empty plan", + ); + assert.equal( + flagOn.plan.credentials.daytonaSecretPlan.candidates.length, + 0, + ); // local_use values still reach sandbox create as plaintext env (by design). assert.equal( flagOn.plan.credentials.modelEnvironment.AWS_ACCESS_KEY_ID, @@ -1307,10 +1349,17 @@ describe("buildRunPlan durableCwd (prefix-derived cwd)", () => { assert.equal(result.ok, true); if (!result.ok) return; - assert.equal(result.plan.workspace.cwd, "/tmp/agenta/mounts/proj-1/mount-abc"); + assert.equal( + result.plan.workspace.cwd, + "/tmp/agenta/mounts/proj-1/mount-abc", + ); // Relay dir is an ephemeral sibling (leaf = cwd basename), NOT inside the durable mount. - assert.ok(!result.plan.workspace.relayDir.startsWith(result.plan.workspace.cwd)); - assert.ok(result.plan.workspace.relayDir.endsWith("/agenta/relay/mount-abc")); + assert.ok( + !result.plan.workspace.relayDir.startsWith(result.plan.workspace.cwd), + ); + assert.ok( + result.plan.workspace.relayDir.endsWith("/agenta/relay/mount-abc"), + ); // createLocalCwd received the durableCwd value. assert.deepEqual(localCwdCalls, ["/tmp/agenta/mounts/proj-1/mount-abc"]); }); @@ -1362,7 +1411,10 @@ describe("buildRunPlan durableCwd (prefix-derived cwd)", () => { assert.equal(result.ok, true); if (!result.ok) return; - assert.equal(result.plan.workspace.cwd, "/tmp/agenta-sandbox-agent-ephemeral"); + assert.equal( + result.plan.workspace.cwd, + "/tmp/agenta-sandbox-agent-ephemeral", + ); assert.deepEqual(localCwdCalls, [undefined]); }); @@ -1567,7 +1619,10 @@ describe("buildRunPlan runtime_provided (subscription) gates", () => { assert.equal(result.ok, true); if (!result.ok) return; assert.equal(result.plan.credentials.credentialMode, "runtime_provided"); - assert.equal(result.plan.workspace.sourcePiAgentDir, "/agenta/harness/pi"); + assert.equal( + result.plan.workspace.sourcePiAgentDir, + "/agenta/harness/pi", + ); }); }); @@ -1784,3 +1839,77 @@ describe("modelConnection validation", () => { if (!result.ok) assert.match(result.error, /modelConnection object/); }); }); + +describe("gateway guidance splice", () => { + const base = { + harness: "pi_core", + messages: [{ role: "user", content: "hi" }], + } as unknown as AgentRunRequest; + const deps = { createLocalCwd: () => "local-cwd" }; + + it("splices the guidance ahead of the authored append prompt for Pi", () => { + const result = buildRunPlan( + { + ...base, + appendSystemPrompt: "Be terse.", + gatewayGuidance: { + text: "## Connected integrations\nuse search_tools", + carrier: "appendSystemPrompt", + }, + } as unknown as AgentRunRequest, + deps, + ); + assert.equal(result.ok, true); + assert.equal( + result.ok && result.plan.prompt.appendSystemPrompt, + "## Connected integrations\nuse search_tools\n\nBe terse.", + ); + }); + + it("carries the guidance alone when nothing is authored on the carrier", () => { + const result = buildRunPlan( + { + ...base, + gatewayGuidance: { text: "guide", carrier: "appendSystemPrompt" }, + } as unknown as AgentRunRequest, + deps, + ); + assert.equal(result.ok, true); + assert.equal(result.ok && result.plan.prompt.appendSystemPrompt, "guide"); + assert.equal(result.ok && result.plan.prompt.hasSystemPrompt, true); + }); + + it("splices into agentsMd for the file-based carrier", () => { + const result = buildRunPlan( + { + ...base, + harness: "claude", + agentsMd: "My rules.", + gatewayGuidance: { text: "guide", carrier: "agentsMd" }, + } as unknown as AgentRunRequest, + deps, + ); + assert.equal(result.ok, true); + assert.equal( + result.ok && result.plan.prompt.agentsMd, + "guide\n\nMy rules.", + ); + // Claude never takes the Pi-only append field. + assert.equal(result.ok && result.plan.prompt.appendSystemPrompt, undefined); + }); + + it("leaves the prompts untouched without a guidance field", () => { + const result = buildRunPlan( + { + ...base, + appendSystemPrompt: "Be terse.", + } as unknown as AgentRunRequest, + deps, + ); + assert.equal(result.ok, true); + assert.equal( + result.ok && result.plan.prompt.appendSystemPrompt, + "Be terse.", + ); + }); +}); diff --git a/services/runner/tests/unit/server.test.ts b/services/runner/tests/unit/server.test.ts index ebe2ccb96b..d452842d53 100644 --- a/services/runner/tests/unit/server.test.ts +++ b/services/runner/tests/unit/server.test.ts @@ -290,7 +290,6 @@ describe("createAgentServer", () => { claude: { state: "login_missing", provider: "anthropic" }, // One Pi mount, one login: both Pi harnesses read it. pi_core: { state: "ready" }, - pi_agenta: { state: "ready" }, }); // The fake credential sitting in the login file the route just read is not on the wire, // and neither is any path. diff --git a/services/runner/tests/unit/session-lifecycle-characterization.test.ts b/services/runner/tests/unit/session-lifecycle-characterization.test.ts index 8b411614e1..8097f298c5 100644 --- a/services/runner/tests/unit/session-lifecycle-characterization.test.ts +++ b/services/runner/tests/unit/session-lifecycle-characterization.test.ts @@ -88,7 +88,11 @@ interface FakeEnv { } interface TurnScript { - approvalPause?: { permissionId: string; toolCallId: string; toolName?: string }; + approvalPause?: { + permissionId: string; + toolCallId: string; + toolName?: string; + }; result?: AgentRunResult; toolCallIds?: string[]; } @@ -188,7 +192,9 @@ function makeEngine(scripts: TurnScript[] = []) { env.approvalGateCount = 1; return { ok: true, stopReason: "paused" }; } - return script.result ?? { ok: true, output: "ok", stopReason: "complete" }; + return ( + script.result ?? { ok: true, output: "ok", stopReason: "complete" } + ); }, async runCold() { return { ok: true, output: "cold", stopReason: "complete" }; @@ -366,13 +372,26 @@ describe("(b) teardown reasons name the failing layer", () => { "capacity-eviction", "shutdown-idle", ] as const) { - assert.equal(teardownDisposition(reason), "stop", `${reason} parks the sandbox`); + assert.equal( + teardownDisposition(reason), + "stop", + `${reason} parks the sandbox`, + ); } }); it("the ordinary delete reasons are unchanged", () => { - for (const reason of ["kill", "failed-turn", "aborted", "shutdown-in-flight"] as const) { - assert.equal(teardownDisposition(reason), "delete", `${reason} deletes the sandbox`); + for (const reason of [ + "kill", + "failed-turn", + "aborted", + "shutdown-in-flight", + ] as const) { + assert.equal( + teardownDisposition(reason), + "delete", + `${reason} deletes the sandbox`, + ); } }); @@ -402,7 +421,11 @@ describe("(b) teardown reasons name the failing layer", () => { "shutdown-in-flight", "shutdown-idle", ]; - assert.equal(everyReason.length, 13, "the TeardownReason union has 13 members"); + assert.equal( + everyReason.length, + 13, + "the TeardownReason union has 13 members", + ); for (const reason of everyReason) { assert.equal( teardownDisposition(reason), @@ -426,7 +449,12 @@ describe("(b) teardown reasons name the failing layer", () => { // credentials are all sound, so the only correct answer is to park it. const { engine, calls } = makeEngine(); const ctx = makeCtx(engine); - await runWithKeepalive(requestWithRevision("rev-1"), undefined, undefined, ctx); + await runWithKeepalive( + requestWithRevision("rev-1"), + undefined, + undefined, + ctx, + ); const env1 = calls.acquiredEnvs[0]; // Turn 2 rewrites the first user message, so the history fingerprint cannot match. @@ -443,7 +471,11 @@ describe("(b) teardown reasons name the failing layer", () => { ctx, ); - assert.equal(calls.acquire, 2, "the conversation is wrong, so the turn still runs cold"); + assert.equal( + calls.acquire, + 2, + "the conversation is wrong, so the turn still runs cold", + ); assert.deepEqual( env1.destroyReasons, ["continuity-invalid"], @@ -462,7 +494,10 @@ describe("(b) teardown reasons name the failing layer", () => { // the stale material still installed. const { engine, calls } = makeEngine(); const ctx = makeCtx(engine); - const withCredential = (secret: string, messages: AgentRunRequest["messages"]) => ({ + const withCredential = ( + secret: string, + messages: AgentRunRequest["messages"], + ) => ({ ...requestWithRevision("rev-1", { messages }), modelConnection: { provider: "openai", @@ -550,7 +585,10 @@ describe("(c) applied state is owned by the environment, never stamped by a requ }; } - async function parkThenResume(resume: AgentRunRequest, parkRevision = "rev-1") { + async function parkThenResume( + resume: AgentRunRequest, + parkRevision = "rev-1", + ) { const { engine, calls } = makeEngine([ { approvalPause: { @@ -599,7 +637,11 @@ describe("(c) applied state is owned by the environment, never stamped by a requ const { ctx, parked, parkedFp } = await parkThenResume(resume, "rev-1"); const reparked = ctx.pool.get(POOL_KEY)!; - assert.equal(reparked.state, "idle", "the resumed turn completed and re-parked"); + assert.equal( + reparked.state, + "idle", + "the resumed turn completed and re-parked", + ); assert.equal( reparked.configFingerprint, @@ -624,7 +666,11 @@ describe("(c) applied state is owned by the environment, never stamped by a requ // desired value. const resume = approveResume("rev-1", { model: "m2" }); const { calls, ctx, parked } = await parkThenResume(resume, "rev-1"); - assert.equal(calls.acquire, 1, "the model change does not evict on the approval branch"); + assert.equal( + calls.acquire, + 1, + "the model change does not evict on the approval branch", + ); const env = calls.acquiredEnvs[0]; assert.equal( @@ -654,12 +700,18 @@ describe("(c) applied state is owned by the environment, never stamped by a requ { role: "user", content: "do X" }, { role: "assistant", - content: [{ type: "tool_call", toolCallId: "tc-gate", toolName: "commit" }], + content: [ + { type: "tool_call", toolCallId: "tc-gate", toolName: "commit" }, + ], }, { role: "user", content: [ - { type: "tool_result", toolCallId: "tc-gate", output: { approved: true } }, + { + type: "tool_result", + toolCallId: "tc-gate", + output: { approved: true }, + }, ], }, { role: "assistant", content: "resumed" }, @@ -693,7 +745,7 @@ describe("(c) applied state is owned by the environment, never stamped by a requ const facets = (tag: string) => Object.fromEntries(FACETS.map((f) => [f, `${tag}-${f}`])) as FacetDigests; - const applied = new AppliedState("fp-m1", facets("m1")); + const applied = new AppliedState("fp-m1", facets("m1"), {}); assert.equal(applied.appliedState.configFingerprint, "fp-m1"); assert.equal(applied.appliedState.generation, 1); assert.equal(applied.appliedState.facets.runtime, "m1-runtime"); @@ -708,21 +760,34 @@ describe("(c) applied state is owned by the environment, never stamped by a requ assert.equal(applied.appliedState.configFingerprint, "fp-m1"); assert.equal(applied.appliedState.facets.runtime, "m1-runtime"); - applied.commitApplied({ configFingerprint: "fp-m2", facets: facets("m2") }); + applied.commitApplied({ + configFingerprint: "fp-m2", + facets: facets("m2"), + fieldDigests: {}, + }); assert.equal(applied.appliedState.configFingerprint, "fp-m2"); assert.equal(applied.appliedState.facets.runtime, "m2-runtime"); assert.equal(applied.appliedState.generation, 2); // Re-applying the same configuration still advances the generation, so "nothing changed" and // "we re-applied" stay distinguishable. - applied.commitApplied({ configFingerprint: "fp-m2", facets: facets("m2") }); + applied.commitApplied({ + configFingerprint: "fp-m2", + facets: facets("m2"), + fieldDigests: {}, + }); assert.equal(applied.appliedState.generation, 3); }); it("for contrast: the IDLE branch does compare the fingerprint and evicts", async () => { const { engine, calls } = makeEngine(); const ctx = makeCtx(engine); - await runWithKeepalive(requestWithRevision("rev-1"), undefined, undefined, ctx); + await runWithKeepalive( + requestWithRevision("rev-1"), + undefined, + undefined, + ctx, + ); await runWithKeepalive( { ...requestWithRevision("rev-1", { diff --git a/services/runner/tests/unit/session-pool.test.ts b/services/runner/tests/unit/session-pool.test.ts index 14f542c92e..20107d4a80 100644 --- a/services/runner/tests/unit/session-pool.test.ts +++ b/services/runner/tests/unit/session-pool.test.ts @@ -25,7 +25,9 @@ const FAKE_FACETS: FacetDigests = Object.fromEntries( ) as FacetDigests; import { approvalDecisionForToolCall, + changedConfigFields, computeCredentialEpoch, + configFieldDigests, configFingerprint, credentialEpochMismatch, credentialEpochValid, @@ -72,7 +74,7 @@ describe("resolvesToLocalProvider (local/remote gate)", () => { // hand the pool a fingerprint of its own. function fakeEnv(configFp = "cfg") { const state = { destroyed: 0, reasons: [] as string[] }; - const applied = new AppliedState(configFp, FAKE_FACETS); + const applied = new AppliedState(configFp, FAKE_FACETS, {}); let done = false; return { state, @@ -257,6 +259,83 @@ describe("configFingerprint", () => { messages: [{ role: "user", content: "hi" }], }; + it("names the changed fields on a config mismatch, values never", () => { + // The `mismatch (config)` eviction line logs WHICH fields differ; each side is a digest + // map, so no config value can reach a log through this path. + const before = configFieldDigests(base); + const after = configFieldDigests({ + ...base, + permissions: { default: "deny" }, + mcpServers: [{ name: "gh" }], + } as unknown as AgentRunRequest); + assert.deepEqual(changedConfigFields(after, before), [ + "mcpServers", + "permissions", + ]); + assert.deepEqual(changedConfigFields(after, undefined), []); + for (const digest of Object.values(after)) { + assert.match(digest, /^[0-9a-f]{64}$/); + } + }); + + it("evicts when the agent artifact changes, and ignores the rest of runContext", () => { + // Audit finding 4: the artifact id selects the agent mount, which is baked at acquire — + // a warm sandbox must never serve a session whose storage folder changed. Every other + // runContext field is per-turn metadata and must never evict (the step-1 rule). + const withContext = (runContext: unknown): AgentRunRequest => + ({ ...base, runContext }) as AgentRunRequest; + const artifactA = withContext({ workflow: { artifact: { id: "art-a" } } }); + assert.notEqual( + configFingerprint(artifactA), + configFingerprint( + withContext({ workflow: { artifact: { id: "art-b" } } }), + ), + "a changed artifact id must evict", + ); + assert.notEqual( + configFingerprint(base), + configFingerprint(artifactA), + "absent-to-present must evict too (the mount appears)", + ); + assert.equal( + configFingerprint(artifactA), + configFingerprint( + withContext({ + workflow: { + artifact: { id: "art-a" }, + revision: { id: "rev-9" }, + variant: { id: "var-9" }, + }, + trace: { trace_id: "xyz" }, + }), + ), + "revision/variant/trace identity stays per-turn metadata", + ); + }); + + it("excludes the derived gateway guidance, so an integration add never evicts", () => { + // The guidance text carries the integration NAMES as examples and refreshes at + // environment build. Hashing it would cold every warm session on each integration add — + // the exact cost the separate field removes. + const a = configFingerprint(base); + const b = configFingerprint({ + ...base, + gatewayGuidance: { + text: "For instance, some of the integrations you have: github, slack.", + carrier: "agentsMd", + }, + } as unknown as AgentRunRequest); + const c = configFingerprint({ + ...base, + gatewayGuidance: { + text: "For instance, some of the integrations you have: github.", + carrier: "agentsMd", + }, + } as unknown as AgentRunRequest); + assert.equal(a, b); + assert.equal(b, c); + }); + it("ignores per-turn volatiles and credential values", () => { const a = configFingerprint({ ...base, @@ -345,8 +424,11 @@ describe("configFingerprint", () => { ); }); - it("changes when resolved model capabilities change", () => { - assert.notEqual( + it("ignores resolved model capabilities (per-turn data that rides with the model)", () => { + // Reversed 2026-08-30 (cold/warm audit finding 2): the modalities are read per turn by the + // attachment chain and change WITH the model, so hashing them refused the live setModel + // route on every cross-modality switch and rebuilt the sandbox for nothing. + assert.equal( configFingerprint(base), configFingerprint({ ...base, @@ -1012,7 +1094,7 @@ describe("SessionPool", () => { it("strict capacity keeps a stopping seat and awaits teardown before inserting", async () => { let releaseTeardown: (() => void) | undefined; let teardownCompleted = false; - const stoppingApplied = new AppliedState("cfg", FAKE_FACETS); + const stoppingApplied = new AppliedState("cfg", FAKE_FACETS, {}); const stoppingEnv = { state: { destroyed: 0, reasons: [] as string[] }, get appliedState() { @@ -1086,7 +1168,7 @@ describe("SessionPool", () => { it("a strict stopping entry cannot be checked out or reparked over", async () => { let releaseTeardown: (() => void) | undefined; - const envApplied = new AppliedState("cfg", FAKE_FACETS); + const envApplied = new AppliedState("cfg", FAKE_FACETS, {}); const environment = { state: { destroyed: 0, reasons: [] as string[] }, get appliedState() { @@ -1133,7 +1215,7 @@ describe("SessionPool", () => { it("non-strict capacity still frees the seat before teardown completes", async () => { let releaseTeardown: (() => void) | undefined; let teardownCompleted = false; - const nonStrictApplied = new AppliedState("cfg", FAKE_FACETS); + const nonStrictApplied = new AppliedState("cfg", FAKE_FACETS, {}); const environment = { state: { destroyed: 0, reasons: [] as string[] }, get appliedState() { @@ -1413,7 +1495,7 @@ describe("SessionPool", () => { // A's destroy is gated: it does not resolve until we release it, standing in for a slow unmount. let releaseADestroy: (() => void) | undefined; const aState = { destroyed: 0, reasons: [] as string[] }; - const aApplied = new AppliedState("cfg", FAKE_FACETS); + const aApplied = new AppliedState("cfg", FAKE_FACETS, {}); const aEnv = { state: aState, get appliedState() { diff --git a/services/runner/tests/unit/subscription-status.test.ts b/services/runner/tests/unit/subscription-status.test.ts index ec0fb0c4e8..f2bcc6b51c 100644 --- a/services/runner/tests/unit/subscription-status.test.ts +++ b/services/runner/tests/unit/subscription-status.test.ts @@ -66,14 +66,10 @@ describe("harnessSubscriptionStatus", () => { state: "not_configured", provider: "anthropic", }); - // Pi is not tied to one provider, so it reports no provider at all — and both Pi harnesses - // answer alike, because they read the same login. + // Pi is not tied to one provider, so it reports no provider at all. assert.deepEqual(await harnessSubscriptionStatus("pi_core", {}), { state: "not_configured", }); - assert.deepEqual(await harnessSubscriptionStatus("pi_agenta", {}), { - state: "not_configured", - }); }); it("treats a whitespace-only mount variable as not configured", async () => { @@ -166,14 +162,10 @@ describe("harnessSubscriptionStatus", () => { ), { state: "ready", provider: "anthropic" }, ); - // One Pi mount, one login file: `pi_core` and `pi_agenta` both read it. const piMount = mount("PI_CODING_AGENT_DIR", "auth.json"); assert.deepEqual(await harnessSubscriptionStatus("pi_core", piMount), { state: "ready", }); - assert.deepEqual(await harnessSubscriptionStatus("pi_agenta", piMount), { - state: "ready", - }); }); it("names the provider families a Pi login holds", async () => { @@ -187,12 +179,10 @@ describe("harnessSubscriptionStatus", () => { }), ); - for (const harness of ["pi_core", "pi_agenta"]) { - assert.deepEqual(await harnessSubscriptionStatus(harness, env), { - state: "ready", - providers: ["anthropic", "openai"], - }); - } + assert.deepEqual(await harnessSubscriptionStatus("pi_core", env), { + state: "ready", + providers: ["anthropic", "openai"], + }); }); it("ignores a login id it has no provider family for", async () => { @@ -344,24 +334,21 @@ describe("subscriptionStatusResponse", () => { assert.equal(response.harnesses.codex.state, "login_unusable"); assert.equal(response.harnesses.claude.state, "ready"); assert.equal(response.harnesses.pi_core.state, "not_configured"); - assert.equal(response.harnesses.pi_agenta.state, "not_configured"); }); - it("gives both Pi harnesses the same answer from the one Pi mount", async () => { + it("reports the Pi states from the one Pi mount", async () => { const piDir = join(root, "pi-agent"); mkdirSync(piDir, { recursive: true }); const env = { PI_CODING_AGENT_DIR: piDir } as NodeJS.ProcessEnv; - // No login file yet: both Pi harnesses say so. + // No login file yet. let response = await subscriptionStatusResponse(env); assert.equal(response.harnesses.pi_core.state, "login_missing"); - assert.equal(response.harnesses.pi_agenta.state, "login_missing"); writeFileSync(join(piDir, "auth.json"), FAKE_LOGIN); response = await subscriptionStatusResponse(env); assert.deepEqual(response.harnesses.pi_core, { state: "ready" }); - assert.deepEqual(response.harnesses.pi_agenta, { state: "ready" }); }); it("serializes nothing but state words and provider names", async () => { diff --git a/services/runner/tests/unit/wire-contract.test.ts b/services/runner/tests/unit/wire-contract.test.ts index 25e13eb35b..dfd9c5b4cc 100644 --- a/services/runner/tests/unit/wire-contract.test.ts +++ b/services/runner/tests/unit/wire-contract.test.ts @@ -50,6 +50,7 @@ const KNOWN_REQUEST_KEYS = [ "toolCallback", "permissions", "gatewayPolicy", + "gatewayGuidance", "systemPrompt", "appendSystemPrompt", "skills", diff --git a/services/uv.lock b/services/uv.lock index b8d2766ae3..c892f3a43b 100644 --- a/services/uv.lock +++ b/services/uv.lock @@ -8,7 +8,7 @@ resolution-markers = [ [[package]] name = "agenta" -version = "0.114.3" +version = "0.114.4" source = { editable = "../sdks/python" } dependencies = [ { name = "agenta-client" }, @@ -72,7 +72,7 @@ dev = [ [[package]] name = "agenta-client" -version = "0.114.3" +version = "0.114.4" source = { editable = "../clients/python" } dependencies = [ { name = "httpx" }, @@ -2356,7 +2356,7 @@ wheels = [ [[package]] name = "services" -version = "0.114.3" +version = "0.114.4" source = { virtual = "." } dependencies = [ { name = "agenta" }, diff --git a/web/ee/package.json b/web/ee/package.json index e70127d51d..2ff931ff62 100644 --- a/web/ee/package.json +++ b/web/ee/package.json @@ -1,6 +1,6 @@ { "name": "@agenta/ee", - "version": "0.114.3", + "version": "0.114.4", "private": true, "engines": { "node": "24.x" diff --git a/web/ee/src/pages/404.tsx b/web/ee/src/pages/404.tsx new file mode 100644 index 0000000000..a1ac3578e7 --- /dev/null +++ b/web/ee/src/pages/404.tsx @@ -0,0 +1,3 @@ +import NotFound from "@agenta/oss/src/pages/404" + +export default NotFound diff --git a/web/mobile/package.json b/web/mobile/package.json index 5f7757ade9..85ce3d8498 100644 --- a/web/mobile/package.json +++ b/web/mobile/package.json @@ -1,6 +1,6 @@ { "name": "@agenta/mobile", - "version": "0.114.3", + "version": "0.114.4", "private": true, "engines": { "node": "24.x" diff --git a/web/mobile/src/components/AgentaLogo.tsx b/web/mobile/src/components/AgentaLogo.tsx index d474761109..84b64f1bb1 100644 --- a/web/mobile/src/components/AgentaLogo.tsx +++ b/web/mobile/src/components/AgentaLogo.tsx @@ -1,27 +1,2 @@ -interface AgentaLogoProps { - className?: string -} - -/** - * Agenta wordmark, inlined from the desktop logo assets. - * - * Inlined rather than served from `public/`: a bare `` misses the app's - * `/m` basePath. The MARK carries the brand accent per theme (the desktop rail's - * `-dark-accent` asset uses #F2F25C on dark, the light asset #1E1C1D); the wordmark stays - * `currentColor` so it follows the surrounding text. - */ -export const AgentaLogo = ({className}: AgentaLogoProps) => ( - - - - -) +/** Agenta wordmark. Inline SVG from @agenta/auth-ui: a `public/` path misses the `/m` basePath. */ +export {AgentaWordmark as AgentaLogo} from "@agenta/auth-ui" diff --git a/web/mobile/src/pages/404.tsx b/web/mobile/src/pages/404.tsx new file mode 100644 index 0000000000..29685e7b38 --- /dev/null +++ b/web/mobile/src/pages/404.tsx @@ -0,0 +1,19 @@ +import {NotFoundScreen} from "@agenta/auth-ui" +import Head from "next/head" +import {useRouter} from "next/router" + +export default function NotFound() { + const router = useRouter() + + return ( + <> + + Page not found · Agenta + + router.back()} + path={router.asPath === "/404" ? undefined : router.asPath} + /> + + ) +} diff --git a/web/oss/package.json b/web/oss/package.json index 830db7e5bf..a47e304818 100644 --- a/web/oss/package.json +++ b/web/oss/package.json @@ -1,6 +1,6 @@ { "name": "@agenta/oss", - "version": "0.114.3", + "version": "0.114.4", "private": true, "engines": { "node": "24.x" diff --git a/web/oss/src/components/AgentChatSlice/AgentChatPanel.tsx b/web/oss/src/components/AgentChatSlice/AgentChatPanel.tsx index 8b4ad3a0d8..0327cdad55 100644 --- a/web/oss/src/components/AgentChatSlice/AgentChatPanel.tsx +++ b/web/oss/src/components/AgentChatSlice/AgentChatPanel.tsx @@ -25,6 +25,7 @@ import { type PendingSessionOpen, } from "@agenta/sessions/state" import {simulatedAgentRunAtomFamily} from "@agenta/shared/state" +import {useSessionShortcuts} from "@agenta/ui/shortcuts" import {paneSlideHoldMs, SplitPane} from "@agenta/ui/ui" import {useAtomValue, useSetAtom, useStore} from "jotai" @@ -42,7 +43,6 @@ import RightPanelSplit from "./components/RightPanel/RightPanelSplit" import SessionHistoryMenu from "./components/SessionHistoryMenu" import ShowConfigPanelButton from "./components/ShowConfigPanelButton" import {useSessionActions} from "./hooks/useSessionActions" -import {useSessionShortcuts} from "./hooks/useSessionShortcuts" import {useReconcileServerSessions} from "./state/projectSessions" import { FILES_PANE_MAX, @@ -244,6 +244,11 @@ const AgentChatPanel = ({entityId}: {entityId: string}) => { const requestRename = useSetAtom(renameSessionRequestAtom) const requestSessionSearch = useSetAtom(sessionSearchRequestAtom) const drawerOpen = useAtomValue(workflowRevisionDrawerOpenAtom) + // Docked Files pane — a full-height sibling of the WHOLE chat column (session bar included), + // like the config pane on the other side: its divider runs to the top and the session bar + // stays confined to the chat. Follows the ACTIVE session (openers set per-session atoms). + const filesPane = useSessionFilesPane(activeId ?? "") + useSessionShortcuts({ sessions, activeId, @@ -284,12 +289,12 @@ const AgentChatPanel = ({entityId}: {entityId: string}) => { () => setConfigPanelCollapsed(!configPanelCollapsed), [configPanelCollapsed, setConfigPanelCollapsed], ), + onToggleFilesPane: useCallback(() => { + if (filesPane.open) filesPane.close() + else filesPane.openPane() + }, [filesPane]), }) - // Docked Files pane — a full-height sibling of the WHOLE chat column (session bar included), - // like the config pane on the other side: its divider runs to the top and the session bar - // stays confined to the chat. Follows the ACTIVE session (openers set per-session atoms). - const filesPane = useSessionFilesPane(activeId ?? "") // Workflow artifact id — the key for the agent's durable `agent-files` mount; the pane's // DriveSessionProvider needs it here because it sits OUTSIDE the per-tab conversations. const artifactId = useAtomValue(workflowMolecule.selectors.workflowId(entityId)) diff --git a/web/oss/src/components/AgentChatSlice/AgentConversation.tsx b/web/oss/src/components/AgentChatSlice/AgentConversation.tsx index 2a18bc4b77..2b3e63b3f9 100644 --- a/web/oss/src/components/AgentChatSlice/AgentConversation.tsx +++ b/web/oss/src/components/AgentChatSlice/AgentConversation.tsx @@ -37,8 +37,10 @@ import {DriveSessionProvider} from "@agenta/entity-ui/drive" import {filesDrawerStagedAtomFamily} from "@agenta/entity-ui/drive" import {buildRenderMap, isPendingClientToolInteraction} from "@agenta/playground" import {simulatedAgentRunAtomFamily} from "@agenta/shared/state" +import {isOverlayOpen} from "@agenta/shared/utils" import {modal} from "@agenta/ui/app-message" import {type RichChatInputHandle} from "@agenta/ui/rich-chat-input" +import {isAltChord} from "@agenta/ui/shortcuts" import {UploadSimple} from "@phosphor-icons/react" import {type FileUIPart, type UIMessage} from "ai" import {useAtomValue, useSetAtom, useStore} from "jotai" @@ -62,7 +64,6 @@ import {useComposerDraft} from "./hooks/useComposerDraft" import {useFirstRunSeed} from "./hooks/useFirstRunSeed" import {useOnboardingChat} from "./hooks/useOnboardingChat" import {useScrollIntent} from "./hooks/useScrollIntent" -import {isAltChord, isOverlayOpen} from "./hooks/useSessionShortcuts" import {useTranscriptScroll} from "./hooks/useTranscriptScroll" import {useTurnInspector} from "./hooks/useTurnInspector" import {useVirtuosoTranscript} from "./hooks/useVirtuosoTranscript" @@ -447,7 +448,9 @@ const AgentConversation = ({ useEffect(() => { if (activeSessionId !== sessionId) return const onKey = (e: KeyboardEvent) => { - if (isOverlayOpen()) return + // Radix cancels Escape for a layer but still lets it reach us, and it never touches + // Alt+G, which only the overlay check catches. + if (e.defaultPrevented || isOverlayOpen()) return // An IME user presses Escape to cancel composition, not to stop the run. if (e.key === "Escape" && !e.isComposing && busyRef.current) { e.preventDefault() @@ -620,8 +623,29 @@ const AgentConversation = ({ ) const handleResend = useCallback( (messageId: string) => { - setStopped(false) - regenerate({messageId}).catch(ignoreStreamRejection) + const msgs = messagesRef.current + const idx = msgs.findIndex((m) => m.id === messageId) + // Same hazard as rewind (#6362 review): regenerating drops the failed assistant + // turn, including any tool that already ran — a retryable model error can land + // AFTER a completed write, and the retry would run the write again. + const sideEffects = idx >= 0 ? sideEffectingToolsInRange(msgs.slice(idx)) : [] + const run = () => { + setStopped(false) + regenerate({messageId}).catch(ignoreStreamRejection) + } + if (sideEffects.length > 0) { + modal.confirm({ + title: "Retry past a tool that already ran?", + content: `${sideEffects.join(", ")} already executed. Retrying re-runs this turn but will NOT undo it.`, + okText: "Retry anyway", + okButtonProps: {danger: true}, + cancelText: "Cancel", + centered: true, + onOk: run, + }) + } else { + run() + } }, [regenerate, setStopped], ) diff --git a/web/oss/src/components/AgentChatSlice/components/AgentMessage.runError.test.tsx b/web/oss/src/components/AgentChatSlice/components/AgentMessage.runError.test.tsx index 25f6046d88..8bc76946ff 100644 --- a/web/oss/src/components/AgentChatSlice/components/AgentMessage.runError.test.tsx +++ b/web/oss/src/components/AgentChatSlice/components/AgentMessage.runError.test.tsx @@ -30,4 +30,43 @@ describe("RunErrorBody", () => { expect(rendered).toContain("Something broke.") expect(rendered).not.toContain("Add your key") }) + + it("offers Try again for a transient credential-delivery failure", () => { + const rendered = text( + undefined} + />, + ) + + expect(rendered).toContain("Try again") + expect(rendered).not.toContain("Add your key") + }) + + it("hides Try again when no retry handler is wired (not the last turn, or busy)", () => { + const rendered = text( + , + ) + + expect(rendered).not.toContain("Try again") + }) + + it("does not offer Try again for a non-transient failure", () => { + const rendered = text( + undefined} + />, + ) + + expect(rendered).not.toContain("Try again") + }) }) diff --git a/web/oss/src/components/AgentChatSlice/components/AgentMessage.tsx b/web/oss/src/components/AgentChatSlice/components/AgentMessage.tsx index 1123593ca1..9e738bc2cc 100644 --- a/web/oss/src/components/AgentChatSlice/components/AgentMessage.tsx +++ b/web/oss/src/components/AgentChatSlice/components/AgentMessage.tsx @@ -75,6 +75,10 @@ interface AgentMessageProps { /** The turn's trace id for a USER message (its paired assistant's trace) — lets the user turn * borrow the run's real start time so it dates from the trace, not this browser's first-seen. */ turnTraceId?: string + /** Re-run this failed turn — the same regenerate wiring as the Stopped → Resend affordance. + * Stable across renders (the message to retry is passed in, not closed over); the parent + * passes it only on the last turn while a retry can actually run, so it gates position. */ + onRetry?: (messageId: string) => void } /** @@ -146,6 +150,13 @@ const STARTER_CREDIT_CODES = new Set([ "starter_credits_program_paused", ]) +/** Transient failure classes where the honest advice is simply to run the turn again. */ +const RETRYABLE_CODES = new Set([ + "credential_delivery_failed", + "starter_credits_unavailable", + "rate_limited", +]) + /** The ONE rule driving both the clamp and the toggle — they can't disagree and hide text (#5350). */ const isBigError = (text: string) => text.length > 240 || text.split("\n").length > 4 @@ -158,11 +169,14 @@ export const RunErrorBody = ({ text, stateKey, code, + onRetry, }: { text: string stateKey: string /** The runner's failure class, when the turn carried one (`data-agent-error`'s `code`). */ code?: string + /** Re-run the failed turn; offered only for the transient classes in RETRYABLE_CODES. */ + onRetry?: () => void }) => { const stored = useAtomValue(expandedValueAtomFamily(stateKey)) const setExpanded = useSetAtom(setExpandedAtom) @@ -170,6 +184,7 @@ export const RunErrorBody = ({ const expanded = stored ?? false const big = isBigError(text) const offerOwnKey = code ? STARTER_CREDIT_CODES.has(code) : false + const offerRetry = !!onRetry && !!code && RETRYABLE_CODES.has(code) return (
@@ -207,10 +222,14 @@ export const RunErrorBody = ({ className="mt-1" onClick={() => requestProviderDrawer(true)} > - {/* TODO(copy: owner) */} Add your key )} + {offerRetry && ( + + )}
) @@ -343,6 +362,7 @@ const AgentMessage = ({ onClientToolOutput, precededByEmptyAssistant = false, turnTraceId, + onRetry, }: AgentMessageProps) => { const openTraceDrawer = useSetAtom(openTraceDrawerAtom) const isUser = message.role === "user" @@ -598,6 +618,7 @@ const AgentMessage = ({ text={errorText || "The agent run failed."} stateKey={errorKey(message.id)} code={runErrorCode} + onRetry={onRetry ? () => onRetry(message.id) : undefined} /> ) diff --git a/web/oss/src/components/AgentChatSlice/components/AgentTurn.tsx b/web/oss/src/components/AgentChatSlice/components/AgentTurn.tsx index db66894621..7393be8704 100644 --- a/web/oss/src/components/AgentChatSlice/components/AgentTurn.tsx +++ b/web/oss/src/components/AgentChatSlice/components/AgentTurn.tsx @@ -87,6 +87,9 @@ const AgentTurn = ({ onClientToolOutput={onClientToolOutput} precededByEmptyAssistant={precededByEmptyAssistant} turnTraceId={turnTraceId} + // A transient run failure offers "Try again" — the same regenerate wiring as the + // Stopped → Resend pair, and gated the same way: last turn only, never while busy. + onRetry={isLast && !resendDisabled ? onResend : undefined} /> {/* Stopped tag + Resend belong only to the LAST assistant turn (the one you cancelled), gated on position so it can never smear onto past turns. Cleared on resend / ask. */} diff --git a/web/oss/src/components/AgentChatSlice/components/OpenFilesPaneButton.tsx b/web/oss/src/components/AgentChatSlice/components/OpenFilesPaneButton.tsx index 390dd055c6..4fbaf8f82b 100644 --- a/web/oss/src/components/AgentChatSlice/components/OpenFilesPaneButton.tsx +++ b/web/oss/src/components/AgentChatSlice/components/OpenFilesPaneButton.tsx @@ -3,6 +3,8 @@ * pane is hidden (it expands leftward from the right edge), gone while shown (the pane header's * own "»" is the collapse, so a second chevron in the bar would be a duplicate). */ +import {shortcutAria} from "@agenta/shared/utils" +import {ShortcutKeys} from "@agenta/ui/shortcuts" import {Button, SimpleTooltip} from "@agenta/ui/ui" import {CaretDoubleLeft} from "@phosphor-icons/react" @@ -16,10 +18,18 @@ export default function OpenFilesPaneButton({sessionId}: {sessionId: string | nu return ( // side="left": the button hugs the page's right edge, and a top-centered tooltip // overflows the viewport for a frame (horizontal-scrollbar flicker). - + + Show files + + } + side="left" + > + ) : null} + + Report + + + + + {/* Quietest text on the page: it is here to be quoted into a bug report. */} +

+ Error 404 + {mounted && path ? ( + <> + {" · "} + {/* Monospaced so an l is tellable from a 1 when retyping the address. */} + {path} + + ) : null} +

+ + ) +} diff --git a/web/packages/agenta-auth-ui/src/auth.css b/web/packages/agenta-auth-ui/src/auth.css index fc1d5a4dce..d0f2a048f8 100644 --- a/web/packages/agenta-auth-ui/src/auth.css +++ b/web/packages/agenta-auth-ui/src/auth.css @@ -410,3 +410,53 @@ opacity: 0.6; cursor: default; } + +/* ── 404 ───────────────────────────────────────────────────────────────────── + The numerals are the page's only oversized type, so they get their own scale + rather than a headline token: the mark stands in for the middle zero, and the + two 4s have to close around it tightly enough to read as one word. */ +.auth-404-digits { + display: flex; + align-items: center; + justify-content: center; + font-family: + var(--font-inter), + -apple-system, + BlinkMacSystemFont, + "Segoe UI", + Roboto, + "Helvetica Neue", + Arial, + sans-serif; + /* Scales with the viewport so the glyph block never outgrows a phone screen. */ + font-size: clamp(96px, 15vw, 196px); + font-weight: 900; + line-height: 1; + letter-spacing: -0.04em; + color: var(--a-heading); +} +/* Sized against the numerals' cap height, not their em box, so the leaf sits on the + same optical line as the 4s. The small negative margins take up the slack in the SVG's + own letterboxing without letting the leaf collide with either 4. */ +.auth-404-mark { + height: 0.7em; + width: auto; + margin: 0 -0.04em; + flex: none; +} + +/* Error code + failed address. The quietest text on the page — it is here to be + quoted into a bug report, not read. */ +.auth-404-code { + color: var(--a-faint); + font-size: 12px; + line-height: 18px; +} + +/* The keycap and surface buttons are full-width by default because the sign-in + column stacks them. Here they sit side by side and size to their labels. */ +.auth-btn-auto { + width: auto; + padding: 0 20px; + text-decoration: none; +} diff --git a/web/packages/agenta-auth-ui/src/index.ts b/web/packages/agenta-auth-ui/src/index.ts index 28d00018e1..10932c53b9 100644 --- a/web/packages/agenta-auth-ui/src/index.ts +++ b/web/packages/agenta-auth-ui/src/index.ts @@ -2,14 +2,17 @@ * @agenta/auth-ui — the sign-in surface's building blocks, extracted from the OSS design * (auth.css carries the scoped brand tokens, light + dark). Plain elements only; flows run * on @agenta/auth; anything app-specific (security widget, post-auth redirect, provider - * transport) arrives through props. Import "@agenta/auth-ui/auth.css" once per app and wrap - * the surface in `.auth-redesign`. + * transport) arrives through props. The 404 page lives here too — the other surface a + * signed-out visitor lands on, built from the same scoped tokens. Import + * "@agenta/auth-ui/auth.css" once per app and wrap the surface in `.auth-redesign`. */ export type {AuthMessage, AuthSecurityAdapter, AuthSuccessPayload} from "./types" export {ShowErrorMessage} from "./ShowErrorMessage" export {AuthDivider} from "./AuthDivider" export {default as AuthSideBanner} from "./AuthSideBanner" export {AuthShell, type AuthShellProps} from "./AuthShell" +export {AgentaMark, AgentaWordmark} from "./AgentaBrand" +export {NotFoundScreen, type NotFoundScreenProps} from "./NotFoundScreen" export { useSignInFlow, type SignInFlow, diff --git a/web/packages/agenta-chat/src/components/ApprovalCard.tsx b/web/packages/agenta-chat/src/components/ApprovalCard.tsx index a2f3df5fee..725cfe252f 100644 --- a/web/packages/agenta-chat/src/components/ApprovalCard.tsx +++ b/web/packages/agenta-chat/src/components/ApprovalCard.tsx @@ -10,7 +10,9 @@ */ import {useEffect, useId, useMemo, useRef, useState} from "react" +import {isOnScreen, isOverlayOpen, shortcutAria} from "@agenta/shared/utils" import {HeightCollapse} from "@agenta/ui/height-collapse" +import {ShortcutKeys} from "@agenta/ui/shortcuts" import {AutosizeTextarea, Button, Checkbox, LoadingButton} from "@agenta/ui/ui" import {CaretRight, ShieldCheck} from "@phosphor-icons/react" @@ -78,6 +80,8 @@ export const ApprovalCard = ({ // The field stays mounted inside the collapse, so focus it explicitly each time it opens; the // rAF waits for the expand to start so focus lands on a laid-out element. const steerInputRef = useRef(null) + // Every visited session stays mounted behind `display: none`, so a hidden card must not answer. + const rootRef = useRef(null) useEffect(() => { if (!steerOpen) return const raf = requestAnimationFrame(() => steerInputRef.current?.focus()) @@ -128,6 +132,9 @@ export const ApprovalCard = ({ const touchCls = touch ? "relative after:absolute after:-inset-x-1 after:-inset-y-2 after:content-['']" : "" + // The keycaps ride on the actions themselves, so the gesture reads without a hover. A touch + // reader has no keyboard, so they earn no space there. + const showKeys = !touch const approve = () => { if (responding) return @@ -151,6 +158,13 @@ export const ApprovalCard = ({ // already no-op while `responding`, so a double-fire is harmless. useEffect(() => { const onKeyDown = (event: KeyboardEvent) => { + // Something on top owns the keyboard. Both halves are load-bearing: Radix cancels + // Escape for a dialog, menu or popover but still lets it reach us, and it never + // touches Cmd+Enter, which only the overlay check catches. + if (event.defaultPrevented || isOverlayOpen()) return + // The listener is on `window`, and a parallel run parks a gate in a session you are + // not looking at. Without this, one Cmd+Enter answered every hidden card too. + if (rootRef.current && !isOnScreen(rootRef.current)) return if (steerOpen) return const approveChord = (event.metaKey || event.ctrlKey) && event.key === "Enter" const denyChord = event.key === "Escape" && !event.metaKey && !event.ctrlKey @@ -173,7 +187,7 @@ export const ApprovalCard = ({ }) return ( -
+
{/* Eyebrow: a quiet cue that a decision is owed, not an error tint. */}
@@ -287,16 +301,30 @@ export const ApprovalCard = ({ loading={responding && firedAction === "deny"} className={touchCls} onClick={deny} + aria-keyshortcuts={shortcutAria("approval.deny")} > {batched && onDenyAll ? "Deny all" : "Deny"} + {/* Decorative: the button's own label already names the action. */} + {showKeys ? ( + + ) : null} {batched ? "Approve all" : "Approve"} + {showKeys ? ( + + ) : null}
diff --git a/web/packages/agenta-chat/src/components/ConnectionDock.tsx b/web/packages/agenta-chat/src/components/ConnectionDock.tsx index f1ab22abb9..7d675fc691 100644 --- a/web/packages/agenta-chat/src/components/ConnectionDock.tsx +++ b/web/packages/agenta-chat/src/components/ConnectionDock.tsx @@ -19,6 +19,7 @@ import { useConnectFlow, useIntegrationIdentity, } from "@agenta/entity-ui/clientTools" +import {isOverlayOpen} from "@agenta/shared/utils" import {Button} from "@agenta/ui/ui" import {Spinner} from "@phosphor-icons/react" import { @@ -499,6 +500,10 @@ const ConnectBody = ({ useEffect(() => { if (!active || !shortcutsEnabled) return const onKeyDown = (event: KeyboardEvent) => { + // Something on top owns the keyboard. Both halves are load-bearing: Radix cancels + // Escape for a dialog, menu or popover but still lets it reach us, and it never + // touches Cmd+Enter, which only the overlay check catches. + if (event.defaultPrevented || isOverlayOpen()) return const commit = (event.metaKey || event.ctrlKey) && event.key === "Enter" const back = event.key === "Escape" && !event.metaKey && !event.ctrlKey if (!commit && !back) return diff --git a/web/packages/agenta-chat/src/components/RecordingBar.tsx b/web/packages/agenta-chat/src/components/RecordingBar.tsx index 3755cb225b..a96a005299 100644 --- a/web/packages/agenta-chat/src/components/RecordingBar.tsx +++ b/web/packages/agenta-chat/src/components/RecordingBar.tsx @@ -1,5 +1,6 @@ import {useEffect, useState} from "react" +import {isOverlayOpen} from "@agenta/shared/utils" import {ComposerSendButton} from "@agenta/ui/rich-chat-input" import {Button, SimpleTooltip} from "@agenta/ui/ui" import {Check, X} from "@phosphor-icons/react" @@ -53,6 +54,10 @@ const RecordingBar = ({ // Esc discards the take (standard for a modal capture). useEffect(() => { const onKey = (e: KeyboardEvent) => { + // Something on top owns the keyboard. Both halves are load-bearing: Radix cancels + // Escape for a dialog, menu or popover but still lets it reach us, and an antd modal + // cancels nothing, so only the overlay check sees that one. + if (e.defaultPrevented || isOverlayOpen()) return if (e.key === "Escape") { e.preventDefault() cancel() diff --git a/web/packages/agenta-chat/src/hooks/usePushToTalk.ts b/web/packages/agenta-chat/src/hooks/usePushToTalk.ts index c4701c95c8..cbaedebe94 100644 --- a/web/packages/agenta-chat/src/hooks/usePushToTalk.ts +++ b/web/packages/agenta-chat/src/hooks/usePushToTalk.ts @@ -1,6 +1,6 @@ import {useEffect, useRef} from "react" -import {isMacPlatform} from "@agenta/shared/utils" +import {isMacPlatform, isOverlayOpen} from "@agenta/shared/utils" /** How long the chord must be held before the mic opens. */ export const PUSH_TO_TALK_ARM_MS = 300 @@ -8,15 +8,6 @@ export const PUSH_TO_TALK_ARM_MS = 300 /** Modifier key names that keep a held chord alive rather than breaking it. */ const CHORD_KEYS = new Set(["Control", "Alt", "AltGraph"]) -/** True while an antd confirm/modal or a Radix dialog owns the screen. No global open-dialog state - * exists to ask, and these dialogs come from `modal.confirm`, so the DOM is the only witness. */ -const isOverlayOpen = (): boolean => - Boolean( - document.querySelector( - '.ant-modal-wrap:not([style*="display: none"]), [role="dialog"][data-state="open"]', - ), - ) - export interface UsePushToTalkParams { enabled: boolean onStart: () => void diff --git a/web/packages/agenta-chat/tests/unit/ApprovalCard.test.tsx b/web/packages/agenta-chat/tests/unit/ApprovalCard.test.tsx index 2c181f3646..c3c64c5f8b 100644 --- a/web/packages/agenta-chat/tests/unit/ApprovalCard.test.tsx +++ b/web/packages/agenta-chat/tests/unit/ApprovalCard.test.tsx @@ -132,8 +132,9 @@ describe("granting a batch", () => { act(() => { box.dispatchEvent(new MouseEvent("click", {bubbles: true})) }) - const approveAll = [...host.querySelectorAll("button")].find( - (node) => node.textContent === "Approve all", + // Prefix, not equality: the button also carries its keycap, so its text is "Approve all⌘↵". + const approveAll = [...host.querySelectorAll("button")].find((node) => + node.textContent?.startsWith("Approve all"), )! act(() => { approveAll.dispatchEvent(new MouseEvent("click", {bubbles: true})) @@ -153,8 +154,8 @@ describe("granting a batch", () => { act(() => { box.dispatchEvent(new MouseEvent("click", {bubbles: true})) }) - const deny = [...host.querySelectorAll("button")].find( - (node) => node.textContent === "Deny", + const deny = [...host.querySelectorAll("button")].find((node) => + node.textContent?.startsWith("Deny"), )! act(() => { deny.dispatchEvent(new MouseEvent("click", {bubbles: true})) @@ -196,6 +197,21 @@ describe("keyboard shortcuts", () => { window.dispatchEvent(new KeyboardEvent("keydown", {bubbles: true, ...init})) }) + /** What Radix does: cancel the key in the capture phase, but let it keep propagating. */ + const pressCancelled = (init: KeyboardEventInit) => + act(() => { + const event = new KeyboardEvent("keydown", { + bubbles: true, + cancelable: true, + ...init, + }) + window.addEventListener("keydown", (e) => e.preventDefault(), { + capture: true, + once: true, + }) + window.dispatchEvent(event) + }) + it("approves on Cmd/Ctrl+Enter and denies on Escape", () => { const responses: {approved: boolean}[] = [] const {cleanup} = mount({ @@ -234,4 +250,124 @@ describe("keyboard shortcuts", () => { expect(responses).toEqual([]) cleanup() }) + + // Radix handles Escape in the capture phase and calls preventDefault, but never + // stopPropagation, so a dialog's Escape still reached this window listener and silently + // denied the gate behind it. Cmd+Enter was never intercepted at all and silently approved it. + it("answers nothing while a dialog owns the screen", () => { + const responses: unknown[] = [] + const {cleanup} = mount({onRespond: () => responses.push(1)}) + + const dialog = document.createElement("div") + dialog.setAttribute("role", "dialog") + dialog.setAttribute("data-state", "open") + document.body.appendChild(dialog) + + press({key: "Escape"}) + press({key: "Enter", metaKey: true}) + expect(responses).toEqual([]) + + // The same two keys answer the gate again the moment the dialog closes. + dialog.remove() + press({key: "Escape"}) + expect(responses).toEqual([1]) + + cleanup() + }) + + // Radix never cancels Cmd+Enter, so only the overlay check can see the menu. Repro: park a + // gate, open the top bar's settings menu, press Cmd+Enter. The gate was approved unseen. + it("answers nothing while a menu owns the screen", () => { + const responses: unknown[] = [] + const {cleanup} = mount({onRespond: () => responses.push(1)}) + + const menu = document.createElement("div") + menu.setAttribute("role", "menu") + menu.setAttribute("data-state", "open") + document.body.appendChild(menu) + + press({key: "Enter", metaKey: true}) + press({key: "Escape"}) + expect(responses).toEqual([]) + + menu.remove() + press({key: "Escape"}) + expect(responses).toEqual([1]) + + cleanup() + }) + + // Every visited session stays mounted behind `display: none`. Two parallel runs both parking a + // gate meant one Cmd+Enter answered the hidden one too. + it("answers nothing while its own session is hidden", () => { + const visible: unknown[] = [] + const hidden: unknown[] = [] + const mountInto = (parent: HTMLElement, sink: unknown[]) => { + const holder = document.createElement("div") + parent.appendChild(holder) + const root = createRoot(holder) + act(() => { + root.render( + sink.push(1)} + onApproveAll={() => undefined} + />, + ) + }) + return () => act(() => root.unmount()) + } + const shown = document.createElement("div") + const offscreen = document.createElement("div") + offscreen.style.display = "none" + document.body.append(shown, offscreen) + + const cleanShown = mountInto(shown, visible) + const cleanHidden = mountInto(offscreen, hidden) + + press({key: "Enter", metaKey: true}) + expect(visible).toEqual([1]) + expect(hidden).toEqual([]) + + cleanShown() + cleanHidden() + shown.remove() + offscreen.remove() + }) + + // antd sets no data-state and leaves its popups mounted, so the guard matches them by class. + it.each([".ant-dropdown", ".ant-select-dropdown", ".ant-popover", ".ant-modal-wrap"])( + "answers nothing while %s is open", + (cls) => { + const responses: unknown[] = [] + const {cleanup} = mount({onRespond: () => responses.push(1)}) + const popup = document.createElement("div") + popup.className = cls.slice(1) + document.body.appendChild(popup) + + press({key: "Enter", metaKey: true}) + press({key: "Escape"}) + expect(responses).toEqual([]) + + popup.remove() + press({key: "Escape"}) + expect(responses).toEqual([1]) + cleanup() + }, + ) + + // Radix cancels Escape in the capture phase and still lets it propagate. Repro: park a gate, + // open any menu, press Escape. The menu closed AND the gate was denied. + it("answers nothing when a menu already cancelled the key", () => { + const responses: unknown[] = [] + const {cleanup} = mount({onRespond: () => responses.push(1)}) + + pressCancelled({key: "Escape"}) + expect(responses).toEqual([]) + + press({key: "Escape"}) + expect(responses).toEqual([1]) + + cleanup() + }) }) diff --git a/web/packages/agenta-chat/tests/unit/assets/rewind.test.ts b/web/packages/agenta-chat/tests/unit/assets/rewind.test.ts index 8297754f79..07408ebe63 100644 --- a/web/packages/agenta-chat/tests/unit/assets/rewind.test.ts +++ b/web/packages/agenta-chat/tests/unit/assets/rewind.test.ts @@ -62,6 +62,21 @@ describe("sideEffectingToolsInRange", () => { expect(sideEffectingToolsInRange(messages)).toEqual([]) }) + it("flags a completed write inside a FAILED turn (the retry range)", () => { + // #6362 review: a retryable model error (rate_limited) can land AFTER a tool already + // wrote. The retry affordance regenerates from the failed assistant message, so the + // range starting AT that message must surface the completed write for the warning. + const failedTurn = { + id: "m1", + role: "assistant", + parts: [ + {type: "tool-create_issue", state: "output-available"}, + {type: "data-agent-error", data: {code: "rate_limited", text: "429"}}, + ], + } as unknown as UIMessage + expect(sideEffectingToolsInRange([failedTurn])).toEqual(["create_issue"]) + }) + it("dedupes repeated tool names across messages", () => { const messages = [ toolMessage("m1", "send_email", "output-available"), diff --git a/web/packages/agenta-entities/src/secret/core/agentModelCandidates.ts b/web/packages/agenta-entities/src/secret/core/agentModelCandidates.ts index 8d520041b0..48b2ae4a90 100644 --- a/web/packages/agenta-entities/src/secret/core/agentModelCandidates.ts +++ b/web/packages/agenta-entities/src/secret/core/agentModelCandidates.ts @@ -34,6 +34,7 @@ export interface BuildAgentModelCandidatesArgs { pairModelSelection?: Record | null } +// "pi_agenta" is a removed experiment; filter it defensively in case an older API still lists it. export const selectableAgentHarnesses = (harnessIds: string[]): string[] => harnessIds.filter((id) => id !== "pi_agenta") diff --git a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/AgentTemplateControl.tsx b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/AgentTemplateControl.tsx index 1aa47a20c3..94d9d90fa3 100644 --- a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/AgentTemplateControl.tsx +++ b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/AgentTemplateControl.tsx @@ -2,17 +2,17 @@ * AgentTemplateControl * * The agent playground's left config panel. It renders the whole agent config as a set - * of collapsible accordion sections (Model, Instructions, Tools, MCP servers, - * Advanced), built on the reusable {@link ConfigAccordionSection} primitive so the same - * pattern can roll out to other config surfaces. + * of collapsible accordion sections (Model, Instructions, Integrations, Subagents, + * MCP servers, Advanced), built on the reusable {@link ConfigAccordionSection} primitive + * so the same pattern can roll out to other config surfaces. * * Dispatched from `x-ag-type: "agent-template"` / `x-ag-type-ref: "agent-template"` (see * SchemaPropertyRenderer). Its `value` IS the agent template (the `parameters.agent` object, * just as the prompt control's value is the prompt template): the portable definition * (instructions/llm/tools/mcps/skills) is FLAT on it, and the execution parts * (harness/runner/sandbox) are nested sub-objects. It reuses the existing schema controls rather - * than inventing new ones: the model selector (GroupedChoiceControl), the agent tool picker - * (AgentToolSelectorPopover + ToolItemControl), the MCP server editor (McpServerItemControl), enum + * than inventing new ones: the model selector (GroupedChoiceControl), the integration and + * subagent lists (ToolManagementList + SubagentList), the MCP server editor (McpServerItemControl), enum * selects (harness, sandbox, permission policy), and a textarea (agents_md). The shape is the * `agent-template` catalog type generated from the SDK model (AgentTemplateSchema in * agenta.sdk.utils.types); the agent service ships a thin `x-ag-type-ref` the playground resolves @@ -37,7 +37,15 @@ import {stripAgentaMetadataDeep} from "@agenta/shared/utils" import {useRecentFlag, type SectionIndicatorTone} from "@agenta/ui/components/presentational" import {useDrillInUI} from "@agenta/ui/drill-in" import {cn} from "@agenta/ui/styles" -import {Cpu, FileText, GraduationCap, Plugs, SlidersHorizontal, Wrench} from "@phosphor-icons/react" +import { + Cpu, + FileText, + GraduationCap, + Plugs, + PuzzlePiece, + Robot, + SlidersHorizontal, +} from "@phosphor-icons/react" import deepEqual from "fast-deep-equal" import {useAtom, useAtomValue, useStore} from "jotai" @@ -53,9 +61,9 @@ import { type AgentTemplateSectionDescriptor, } from "./agentTemplate/AgentTemplateSectionList" import {countSummary} from "./agentTemplate/agentTemplateUtils" -import {AgentToolSelectorPopover} from "./agentTemplate/AgentToolSelectorPopover" import {ConfigItemList} from "./agentTemplate/ConfigItemList" import {IntegrationPermissionDrawer} from "./agentTemplate/IntegrationPermissionDrawer" +import {toolReferenceSlug} from "./agentTemplate/itemDescriptors" import {ITEM_KINDS, type ItemKind} from "./agentTemplate/itemKinds" import {InstructionsFileRow, type ItemRowStatus} from "./agentTemplate/ItemRow" import {SectionAddButton} from "./agentTemplate/SectionAddButton" @@ -66,7 +74,16 @@ import { type PanelSectionKey, } from "./agentTemplate/sectionChanges" import {SectionTitleBadge} from "./agentTemplate/SectionTitleBadge" -import {ToolManagementList} from "./agentTemplate/ToolManagementList" +import { + ConnectedSubagentList, + SubagentDrawerContainer, +} from "./agentTemplate/SubagentDrawerContainer" +import {SubagentHeaderIcon, SubagentOpenAgentButton} from "./agentTemplate/SubagentHeader" +import { + selectSubagentTools, + SubagentList, + ToolManagementList, +} from "./agentTemplate/ToolManagementList" import {useAgentTools} from "./agentTemplate/useAgentTools" import {useConfigItemDrawer} from "./agentTemplate/useConfigItemDrawer" import {useModelHarness} from "./agentTemplate/useModelHarness" @@ -77,16 +94,14 @@ import {JsonObjectEditor} from "./JsonObjectEditor" import {SectionDrawer} from "./SectionDrawer" import { findIntegrationRow, - isHarnessBuiltinTool, integrationRowConnection, + integrationRowIndices, parseGatewayEntry, type GatewayConnectionTarget, type GatewayEntry, type IntegrationRow, - type ToolObj, } from "./toolUtils" import {useAgentTriggers} from "./TriggerManagementSection" -import {WorkflowReferenceSelector} from "./WorkflowReferenceSelector" // Tooltip copy for the config-panel draft/validation indicators. const INVALID_ITEM_TIP: Record = { @@ -94,10 +109,15 @@ const INVALID_ITEM_TIP: Record = { mcp: "This server is missing its name or URL.", skill: "This skill is missing its name.", } +/** The schema field a section's change marks come from. Integrations and Subagents share `tools`. */ +const changeFieldFor = (sectionKey: string): string => + sectionKey === "subagents" ? "tools" : sectionKey + const DRAFT_TIP: Record = { "model-harness": "Unsaved model or harness changes.", instructions: "Unsaved instruction changes.", - tools: "Unsaved tool changes.", + tools: "Unsaved integration changes.", + subagents: "Unsaved subagent changes.", mcp: "Unsaved MCP server changes.", skills: "Unsaved skill changes.", advanced: "Unsaved advanced-setting changes.", @@ -138,7 +158,7 @@ const ModelHarnessSectionBody = ({ // The four list sections whose open-state is controlled so the accordion can auto-expand when // the agent populates them (see `useAutoExpandOnPopulate`). -const CONTROLLED_SECTION_KEYS = new Set(["tools", "mcp", "skills", "triggers"]) +const CONTROLLED_SECTION_KEYS = new Set(["tools", "subagents", "mcp", "skills", "triggers"]) export const AgentTemplateControl = memo(function AgentTemplateControl({ schema, @@ -343,7 +363,8 @@ export const AgentTemplateControl = memo(function AgentTemplateControl({ const agentChangedKeys = sectionChanges.agent?.panelKeys ?? null const agentChangeIndicator = useCallback( (sectionKey: string) => { - if (!agentChangedKeys?.has(sectionKey as PanelSectionKey)) return undefined + if (!agentChangedKeys?.has(changeFieldFor(sectionKey) as PanelSectionKey)) + return undefined const version = sectionChanges.agentVersion return { tone: "agent" as const, @@ -401,12 +422,8 @@ export const AgentTemplateControl = memo(function AgentTemplateControl({ // Tool add/remove (inline function, builtin, gateway, workflow reference) lives in its own hook. const { tools, - handleAddTool, handleAddWorkflowReference, - handleRemoveToolByName, - handleRemoveBuiltinTool, - selectedToolNames, - referenceableWorkflows, + handleRemoveReferenceBySlug, integrationRows, setIntegrationConnection, setIntegrationPermissions, @@ -441,11 +458,21 @@ export const AgentTemplateControl = memo(function AgentTemplateControl({ [permissionTarget, integrationRows], ) - // Legacy harness built-in entries render nowhere (ToolManagementList drops them), so the - // header count and the section's open state must ignore them too. - const visibleToolCount = useMemo( - () => tools.filter((tool) => !isHarnessBuiltinTool(tool)).length, - [tools], + // The subagents, each keeping its index in `tools`: edit and remove address it by index. + const subagentTools = useMemo( + () => selectSubagentTools(tools, integrationRows), + [tools, integrationRows], + ) + // Each section counts only the kind it renders; what renders nowhere is counted nowhere. + const integrationCount = integrationRows.length + const subagentCount = subagentTools.length + // What the picker marks as added, read from the saved tools rather than the picker's state. + const savedSubagentSlugs = useMemo( + () => + subagentTools + .map(({item}) => toolReferenceSlug(item)) + .filter((s): s is string => Boolean(s)), + [subagentTools], ) // External HTTP MCP servers from the saved agent template. @@ -468,10 +495,11 @@ export const AgentTemplateControl = memo(function AgentTemplateControl({ [openCreate], ) - // Controlled open-state for the four list sections so the accordion can react to the agent + // Controlled open-state for the list sections so the accordion can react to the agent // populating a section. Seeded once from the initial counts; the edge hook below flips it. const [sectionOpen, setSectionOpen] = useState>(() => ({ - tools: visibleToolCount > 0, + tools: integrationCount > 0, + subagents: subagentCount > 0, mcp: mcpServers.length > 0, skills: skills.length > 0, triggers: triggerCount > 0, @@ -483,12 +511,13 @@ export const AgentTemplateControl = memo(function AgentTemplateControl({ ) const sectionCounts = useMemo( () => ({ - tools: visibleToolCount, + tools: integrationCount, + subagents: subagentCount, mcp: mcpServers.length, skills: skills.length, triggers: triggerCount, }), - [visibleToolCount, mcpServers.length, skills.length, triggerCount], + [integrationCount, subagentCount, mcpServers.length, skills.length, triggerCount], ) useAutoExpandOnPopulate(sectionCounts, setSectionOpenByKey) @@ -742,9 +771,23 @@ export const AgentTemplateControl = memo(function AgentTemplateControl({ if (mh.modelUnsupported) return "The selected model isn't available on this harness." return null } + if (key === "subagents") { + return subagentTools.some(({item}) => + ITEM_KINDS.tool.draftInvalid(item as Record), + ) + ? "A subagent is missing its name." + : null + } if (key === "tools") { - if (tools.some((t) => ITEM_KINDS.tool.draftInvalid(t as Record))) - return "A tool is missing its name." + // Integration entries only: anything else has its own header or renders nowhere. + const owned = new Set(integrationRows.flatMap(integrationRowIndices)) + if ( + tools.some( + (t, i) => + owned.has(i) && ITEM_KINDS.tool.draftInvalid(t as Record), + ) + ) + return "An integration is missing its name." if (toolResolutionSummary.unresolved > 0) return "A connected-app tool couldn't be resolved — its action or connection may have been renamed or removed." return null @@ -774,7 +817,7 @@ export const AgentTemplateControl = memo(function AgentTemplateControl({ if (invalid) return {tone: "invalid", tooltip: invalid} const incomplete = sectionIncompleteTip(key) if (incomplete) return {tone: "incomplete", tooltip: incomplete} - if (draftSectionKeys.has(key as PanelSectionKey)) + if (draftSectionKeys.has(changeFieldFor(key) as PanelSectionKey)) return { tone: "draft", tooltip: DRAFT_TIP[key] ?? "Unsaved changes.", @@ -801,27 +844,15 @@ export const AgentTemplateControl = memo(function AgentTemplateControl({ ? {tone: "edited", label: "Edited", tooltip: "Edited — not saved yet."} : undefined - // Shared props for the tool picker, so the in-body popover and the header quick-add trigger - // drive the same add flow. - const toolSelectorProps = { - onAddTool: handleAddTool, - onRemoveTool: handleRemoveToolByName, - onRemoveBuiltinTool: handleRemoveBuiltinTool, - selectedToolNames, - selectedTools: tools as ToolObj[], - existingToolCount: tools.length, - gatewayTools, - onReferenceWorkflow: workflowReference?.enabled - ? () => { - // Opening the picker is the point the workflow list is actually needed — activate - // the (lazy) bridge so it resolves now instead of on every playground load. - workflowReference.activate?.() - setReferenceSelectorOpen(true) - } - : undefined, - // Route the integration row to the agent-scoped drawer instead of the shared global catalog. - onOpenIntegration: gatewayTools?.enabled ? openIntegration : undefined, - } + // Opening the picker is where the workflow list is needed, so activate the lazy bridge here. + const openSubagentSelector = workflowReference?.enabled + ? () => { + workflowReference.activate?.() + setReferenceSelectorOpen(true) + } + : undefined + // The Integrations add button. Routes to the agent-scoped drawer, not the shared global catalog. + const openIntegrationDrawer = gatewayTools?.enabled ? openIntegration : undefined // Compact "+" for a section header's `extra` slot. The header keeps a uniform height regardless // of this button — ConfigAccordionSection collapses the extra slot's vertical footprint (see its @@ -830,6 +861,19 @@ export const AgentTemplateControl = memo(function AgentTemplateControl({ ) + // Shared by both subagent-list branches, which differ only by wrapper. + const subagentListProps = { + entries: subagentTools, + openEdit, + removeItem, + closeEditor, + disabled, + statusFor: toolStatusFor, + emptyAdd: openSubagentSelector ? ( + + ) : undefined, + } + // The inline "what changed" body for the drawer-backed Advanced section. Null when the section // is clean, which is what keeps it a plain drawer-opening row. const advancedChangeBody = changeBodyFor("advanced") @@ -883,46 +927,65 @@ export const AgentTemplateControl = memo(function AgentTemplateControl({
), }, - hasTools && { - key: "tools", - icon: , - title: fieldTitle("tools", "Tools"), - summary: countSummary(visibleToolCount, "tool"), - indicator: sectionIndicator("tools"), - extra: !disabled ? ( - } - /> - ) : undefined, - defaultOpen: visibleToolCount > 0, - content: ( - { - removeIntegration(row) - closeEditor() - }} - // The empty-state add is the same popover as the header +. - emptyAdd={ - } - /> - } - /> - ), - }, + // Connected apps. Shares the `tools` indicator key with Subagents. + hasTools && + (Boolean(openIntegrationDrawer) || integrationCount > 0) && { + key: "tools", + icon: , + title: "Integrations", + summary: countSummary(integrationCount, "integration"), + indicator: sectionIndicator("tools"), + // One action, so the header plus opens the drawer directly instead of a menu. + extra: + !disabled && openIntegrationDrawer + ? headerAddButton("Add integration", openIntegrationDrawer) + : undefined, + defaultOpen: integrationCount > 0, + content: ( + { + removeIntegration(row) + closeEditor() + }} + // The empty-state add opens the header's drawer, and hides when there is none. + emptyAdd={ + openIntegrationDrawer ? ( + + ) : undefined + } + /> + ), + }, + // The agents this agent can call, saved as `{type: "reference"}` in the same `tools` array. + hasTools && + (Boolean(openSubagentSelector) || subagentCount > 0) && { + key: "subagents", + icon: , + title: "Subagents", + summary: countSummary(subagentCount, "subagent"), + indicator: sectionIndicator("subagents"), + extra: + !disabled && openSubagentSelector + ? headerAddButton("Add subagent", openSubagentSelector) + : undefined, + defaultOpen: subagentCount > 0, + // Only the connected list can resolve a reference's type and mark a non-agent. + content: workflowReference?.enabled ? ( + + ) : ( + + ), + }, hasMcp && { key: "mcp", icon: , @@ -990,6 +1053,17 @@ export const AgentTemplateControl = memo(function AgentTemplateControl({ const lastEditingRef = useRef(editing) if (editing) lastEditingRef.current = editing const shownEditing = editing ?? lastEditingRef.current + + // Resolved in children, which mount only when the host supplies a bridge: a hook reached + // through an optional member cannot be called from here without breaking hook order. + const editingSubagentSlug = + shownEditing?.kind === "tool" ? (toolReferenceSlug(draft) ?? "") : "" + const subagentHeaderIcon = workflowReference ? ( + + ) : undefined + const subagentHeaderAction = workflowReference ? ( + + ) : undefined const lastInstructionRef = useRef(editingInstruction) if (editingInstruction) lastInstructionRef.current = editingInstruction const shownInstruction = editingInstruction ?? lastInstructionRef.current @@ -1012,28 +1086,37 @@ export const AgentTemplateControl = memo(function AgentTemplateControl({ const readOnly = disabled || def.isReadOnly(draft) const Form = def.FormView const itemKey = `${shownEditing.kind}-${shownEditing.mode}-${shownEditing.index}` - // Skills state their identity in the form, so the drawer drops the icon, - // badge, subtitle and footer note (rows still show them). + // Skills state their identity in their own form; the drawer drops its chrome. const bareChrome = shownEditing.kind === "skill" + const isSubagent = Boolean(def.statesOwnIdentity?.(draft)) return ( {workflowReference?.enabled && ( - setReferenceSelectorOpen(false)} - workflows={referenceableWorkflows} bridge={workflowReference} - onSelect={(payload) => { - void handleAddWorkflowReference(payload) - setReferenceSelectorOpen(false) + revisionId={revisionId} + savedSlugs={savedSubagentSlugs} + onAdd={handleAddWorkflowReference} + onRemoveSlug={(slug) => { + handleRemoveReferenceBySlug(slug) + closeEditor() }} /> )} diff --git a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/ConfigItemDrawer.tsx b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/ConfigItemDrawer.tsx index 770fce954f..910fa57a19 100644 --- a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/ConfigItemDrawer.tsx +++ b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/ConfigItemDrawer.tsx @@ -82,6 +82,12 @@ export interface ConfigItemDrawerProps { json: ReactNode /** Hide the Form/JSON toggle and show JSON only (e.g. items with no structured form). */ jsonOnly?: boolean + /** Hide the Form/JSON toggle and keep the FORM. For an item whose raw shape is an internal + * detail the reader has no reason to edit. */ + formOnly?: boolean + /** Header action shown where the Form/JSON toggle would be, for an item that has no toggle + * and does have an action of its own (a subagent's "Open agent" link). */ + headerExtra?: ReactNode /** Drawer width in px. @default 600 */ width?: number /** Read-only mode: disables the toggle and the Save action. */ @@ -110,11 +116,13 @@ export function ConfigItemDrawer({ form, json, jsonOnly = false, + formOnly = false, + headerExtra, width = 600, disabled = false, contentFlush = false, }: ConfigItemDrawerProps) { - const effectiveView = jsonOnly ? "json" : view + const effectiveView = jsonOnly ? "json" : formOnly ? "form" : view // Flush layout only helps the form; keep the JSON editor padded and independently scrollable. const flushForm = contentFlush && effectiveView === "form" @@ -155,7 +163,9 @@ export function ConfigItemDrawer({ } extra={ - jsonOnly ? null : ( + jsonOnly || formOnly ? ( + (headerExtra ?? null) + ) : ( (null) -const editReferenceRevisionAdapter = createWorkflowRevisionAdapter({ - workflowIdAtom: editRefWorkflowIdAtom, -}) - -function isRecord(value: unknown): value is Record { - return typeof value === "object" && value !== null && !Array.isArray(value) -} - -function countProps(schema: Record | null): number { - return schema && isRecord(schema.properties) ? Object.keys(schema.properties).length : 0 -} +import {normalizeSubagentReference} from "./agentTemplate/subagentReference" +import {useIntegrationLogos} from "./hooks/useIntegrationLogos" +import {ProviderLogo} from "./sectionGroups" export interface ReferenceToolFormViewProps { value: Record @@ -50,203 +19,214 @@ export interface ReferenceToolFormViewProps { disabled?: boolean } -/** - * The editable Reference-by axis. Isolated so the bridge hooks (`useWorkflowEnvironments`) are only - * mounted when the bridge is available. Commits binding changes onto the reference tool immediately. - */ -function ReferenceBindingEditor({ - tool, - onChange, - bridge, - workflow, -}: { - tool: Record - onChange: (next: Record) => void - bridge: WorkflowReferenceBridge - workflow: WorkflowReferenceUI | null -}) { - const setWorkflowId = useSetAtom(editRefWorkflowIdAtom) - useEffect(() => { - setWorkflowId(workflow?.id ?? null) - return () => setWorkflowId(null) - }, [workflow?.id, setWorkflowId]) - - const [bindMode, setBindMode] = useState<"revision" | "environment">( - tool.ref_by === "environment" ? "environment" : "revision", - ) - const [version, setVersion] = useState( - typeof tool.version === "string" ? tool.version : undefined, - ) - const [environment, setEnvironment] = useState( - typeof tool.environment === "string" ? tool.environment : undefined, - ) - // Selected variant id — kept even when following its latest (no pinned version). - const [variant, setVariant] = useState( - typeof tool.variant_id === "string" ? tool.variant_id : undefined, +/** One row of the configuration card: a muted label column, then the value. */ +function DetailRow({label, children}: {label: string; children: React.ReactNode}) { + return ( +
+ + {label} + +
{children}
+
) +} - const {environments, isLoading} = bridge.useWorkflowEnvironments(workflow) - - // Rebuild the reference tool from the current binding, clearing the now-irrelevant axis field. - const commit = ( - mode: "revision" | "environment", - ver?: string, - env?: string, - varId?: string, - ) => { - const next = {...tool} - if (mode === "revision") { - next.ref_by = "variant" - if (varId) next.variant_id = varId - else delete next.variant_id - if (ver) next.version = ver - else delete next.version - delete next.environment - } else { - next.ref_by = "environment" - if (env) next.environment = env - else delete next.environment - delete next.version - delete next.variant_id - } - onChange(next) - } +/** Muted text for a row with nothing in it, worded like the picker's own empty line. */ +const Empty = ({children}: {children: React.ReactNode}) => ( + {children} +) +/** The agent's instruction file, clamped behind a fade and scrolled inside a fixed well. */ +function Instructions({file}: {file: NonNullable}) { + const [open, setOpen] = useState(false) return ( - { - setBindMode(mode) - commit(mode, version, environment, variant) - }} - revisionAdapter={editReferenceRevisionAdapter} - revisionPlaceholder={version ? `v${version}` : "Latest revision"} - onRevisionSelect={(sel: WorkflowRevisionSelectionResult) => { - const isRevision = - Boolean(sel.metadata.variantId) && sel.id !== sel.metadata.variantId - const ver = isRevision ? String(sel.metadata.revision) : undefined - const varId = sel.metadata.variantId ? String(sel.metadata.variantId) : undefined - setVersion(ver) - setVariant(varId) - commit("revision", ver, environment, varId) - }} - revisionHint="Pin one variant + revision, or pick a variant to follow its latest." - envOptions={environments.map((env) => ({value: env.slug, label: env.name || env.slug}))} - envLoading={isLoading} - environmentSlug={environment} - onEnvironmentChange={(slug) => { - setEnvironment(slug) - commit("environment", version, slug) - }} - envNotFound={ - isLoading ? ( - - ) : ( - - No environments deployed - - ) - } - envHint="Calls whatever revision is deployed in the chosen environment." - /> +
+
+ + {file.fileName} + + Markdown · {file.wordCount.toLocaleString()} words + +
+
+
+ {file.text} +
+ {/* The fade says "there is more" without a second control competing with the link. */} + {open ? null : ( +
+ )} +
+ +
) } -/** One-line summary of the current binding (collapsed section preview + read-only fallback). */ -function bindingSummary(tool: Record): string { - if (tool.ref_by === "environment") { - return typeof tool.environment === "string" - ? `Deployed in ${tool.environment}` - : "A deployed environment" - } - return typeof tool.version === "string" ? `Pinned to v${tool.version}` : "Latest revision" +export function ReferenceToolFormView(props: ReferenceToolFormViewProps) { + const {workflowReference} = useDrillInUI() + // Split so the bridge's hook is never called through an optional member: each branch is its + // own component, so React remounts rather than shifting hook order. + return workflowReference ? ( + + ) : ( + + ) } -/** Read-only binding summary shown when the workflow-reference bridge isn't available. */ -function ReadOnlyBinding({tool}: {tool: Record}) { - return

{bindingSummary(tool)}

+function ConnectedReferenceToolFormView({ + bridge, + ...props +}: ReferenceToolFormViewProps & {bridge: WorkflowReferenceBridge}) { + const slug = typeof props.value?.slug === "string" ? props.value.slug : "" + const {detail, loading} = bridge.useSubagentDetail(slug) + return } -export function ReferenceToolFormView({value, onChange, disabled}: ReferenceToolFormViewProps) { - const tool = (value ?? {}) as Record - const slug = typeof tool.slug === "string" ? tool.slug : "" - const description = typeof tool.description === "string" ? tool.description : "" - const inputSchema = isRecord(tool.input_schema) - ? (tool.input_schema as Record) - : null - - const {workflowReference} = useDrillInUI() - // Displaying an existing reference needs the workflow list to resolve its name — activate the - // (lazy) bridge. Configs with no reference never mount this view, so they never pull the list. - useEffect(() => { - if (slug) workflowReference?.activate?.() - }, [slug, workflowReference]) - const workflow = useMemo( - () => workflowReference?.workflows.find((w) => w.slug === slug) ?? null, - [workflowReference, slug], +function SubagentDetailPanel({ + value, + onChange, + disabled, + detail, + loading, +}: ReferenceToolFormViewProps & {detail: SubagentDetail | null; loading: boolean}) { + const tool = value ?? {} + + // The bridge knows WHICH apps connect; their logos come from the catalog, which needs a component. + const appKeys = useMemo( + () => (detail?.integrations ?? []).map((a) => a.key), + [detail?.integrations], ) + const appByKey = useIntegrationLogos(appKeys) - const setDescription = (next: string) => onChange({...tool, description: next}) + const description = typeof tool.description === "string" ? tool.description : "" + const ProviderIcon = detail?.provider ? getProviderIcon(detail.provider) : null return ( -
-
- } - title="Details" - > - -
- {slug} - -
- - - - setDescription(e.target.value)} - autoSize={{minRows: 2, maxRows: 6}} - placeholder="What this tool does and when the agent should call it" - aria-label="Description" - disabled={disabled} - /> - - - - } - title="Schema" - summary={`Inputs · ${countProps(inputSchema)}`} - summaryCollapsedOnly - > -
- -
- +
+
+ Description + + onChange(normalizeSubagentReference({...tool, description: e.target.value})) + } + aria-label="Subagent description" + /> + + + The agent reads this description to decide when to call this subagent. Edit it + to change when it is used. + +
- } - title="Reference by" - summary={bindingSummary(tool)} - summaryCollapsedOnly - > - {workflowReference?.enabled && !disabled ? ( - +
+
+ Configuration + + + Read-only + + + Managed on the agent itself + +
+
+ {loading && !detail ? ( +
+ + + +
) : ( - + <> + + {detail?.model ? ( + + + {ProviderIcon ? ( + + ) : ( + + )} + + + {detail.model} + + + ) : ( + No model + )} + + + {detail?.instructions ? ( + + ) : ( + No instructions + )} + + + {detail?.integrations.length ? ( +
+ {detail.integrations.map((app) => { + const catalog = appByKey.get(app.key) + return ( + + + + {catalog?.name ?? app.key} + + {app.permission ? ( + + {app.permission} + + ) : null} + + ) + })} +
+ ) : ( + No connected apps + )} + + + {detail?.skills.length ? ( +
+ {detail.skills.map((skill) => ( + + {skill} + + ))} +
+ ) : ( + No skills + )} + + )} - +
) diff --git a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/WorkflowReferenceSelector.tsx b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/WorkflowReferenceSelector.tsx deleted file mode 100644 index 232a93cbd1..0000000000 --- a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/WorkflowReferenceSelector.tsx +++ /dev/null @@ -1,648 +0,0 @@ -/** - * WorkflowReferenceSelector - * - * Two-panel master/detail drawer for referencing a workflow as an agent tool - * (`type:"reference"`, #4860). Left rail: a searchable, type-badged list with present-only - * filter chips. Right panel: the selected workflow's detail — description, the tool name it's - * exposed as, the resolved input schema, and the axis controls (by variant/version or by - * environment) — then Add. When nothing is selected the detail panel shows a resting hint - * rather than dead space. - * - * Styling uses antd semantic tokens (`--ag-color*`) + antd `Tag` (theme-aware) only — dark-safe. - * Built on the shared `EnhancedDrawer`. - */ -import {useEffect, useMemo, useState} from "react" - -import {ConfigAccordionSection, CopyButton} from "@agenta/ui/components/presentational" -import {EnhancedDrawer} from "@agenta/ui/drawer" -import type { - WorkflowConfigPart, - WorkflowReferenceBridge, - WorkflowReferencePayload, - WorkflowReferenceType, - WorkflowReferenceUI, -} from "@agenta/ui/drill-in" -import { - AutosizeTextarea, - Badge, - type BadgeProps, - EmptyState, - InputAffix, - Segmented, - Skeleton, - Spinner, -} from "@agenta/ui/ui" -import { - GitBranch, - GraphIcon, - HandPointing, - MagnifyingGlass, - SlidersHorizontal, - TreeStructure, -} from "@phosphor-icons/react" -import {atom, useSetAtom} from "jotai" - -import {DrawerFooter} from "../../drawers/shared/DrawerFooter" -import {SectionRail} from "../../drawers/shared/SectionRail" -import {RunVersionField} from "../../gatewayTrigger/drawers/shared/RunVersionField" -import {createWorkflowRevisionAdapter, type WorkflowRevisionSelectionResult} from "../../selection" - -import {CodeEditor, type CodeEditorLanguage} from "./CodeEditor" -import {SchemaTree} from "./SchemaTree" - -export interface WorkflowReferenceSelectorProps { - open: boolean - onClose: () => void - /** Workflows available to reference (the caller filters out already-referenced ones). */ - workflows: WorkflowReferenceUI[] - /** Supplies the per-workflow revision, environment, and input-schema lookups. */ - bridge: WorkflowReferenceBridge - /** Emit the chosen reference (axis + slug + version/environment). */ - onSelect: (payload: WorkflowReferencePayload) => void -} - -// Workflow-scoped revision picker (2-level Variant → Revision) for the shared RunVersionField. -// The atom is synced to the selected workflow so the cascader is scoped to it. -const refWorkflowIdAtom = atom(null) -const referenceRevisionAdapter = createWorkflowRevisionAdapter({workflowIdAtom: refWorkflowIdAtom}) - -// Badge preset hues (the antd preset Tag colours, palette-driven so they flip light↔dark). -const TYPE_BADGE: Record = { - agent: {color: "purple", label: "agent"}, - chat: {color: "blue", label: "chat"}, - completion: {color: "cyan", label: "completion"}, - custom: {color: "gold", label: "custom"}, - evaluator: {color: "green", label: "evaluator"}, -} - -// Filter by the workflow's actual type, so the chips read the same as the row badges. -type TypeFilter = "all" | WorkflowReferenceType - -const TYPE_FILTER_ORDER: WorkflowReferenceType[] = [ - "completion", - "chat", - "agent", - "custom", - "evaluator", -] - -function capitalize(value: string): string { - return value.charAt(0).toUpperCase() + value.slice(1) -} - -function isRecord(value: unknown): value is Record { - return typeof value === "object" && value !== null && !Array.isArray(value) -} - -const noop = () => {} - -// Contained render of a Configuration part: the prompt "Messages" render as a role-tagged list; -// plain text wraps in a box; code/JSON use the app's read-only code editor. Long content never grows -// the panel (max-height + internal scroll). -function ConfigPartContent({part}: {part: WorkflowConfigPart}) { - if (part.kind === "messages" && part.messages) { - return ( -
- {part.messages.map((message, i) => ( -
- - {capitalize(message.role)} - -
- {message.content} -
-
- ))} -
- ) - } - if (part.kind === "text") { - return ( -
- {part.content} -
- ) - } - const language: CodeEditorLanguage = - part.kind === "json" ? "json" : ((part.language ?? "code") as CodeEditorLanguage) - return ( -
- -
- ) -} - -function TypeBadge({type, label}: {type: WorkflowReferenceType | undefined; label?: string}) { - if (!type) return null - const cfg = TYPE_BADGE[type] - return ( - - {label || cfg.label} - - ) -} - -export function WorkflowReferenceSelector({ - open, - onClose, - workflows, - bridge, - onSelect, -}: WorkflowReferenceSelectorProps) { - const [search, setSearch] = useState("") - const [filter, setFilter] = useState("all") - const [selected, setSelected] = useState(null) - // bindMode mirrors RunVersionField: "revision" (pin a variant+revision) | "environment" (deployed). - const [bindMode, setBindMode] = useState<"revision" | "environment">("revision") - const [version, setVersion] = useState(undefined) - // Selected variant id — kept even when following its latest (no pinned version). - const [variant, setVariant] = useState(undefined) - const [environment, setEnvironment] = useState(undefined) - // Tool description the model sees — defaults to the workflow's own, editable before adding. - const [description, setDescription] = useState("") - const [inputSchema, setInputSchema] = useState | null>(null) - const [outputSchema, setOutputSchema] = useState | null>(null) - const [schemaLoading, setSchemaLoading] = useState(false) - // Schema section's left-rail selection (Inputs / Outputs). Outputs appears only when the - // bridge resolves an output schema; otherwise the rail shows Inputs alone. - const [schemaTab, setSchemaTab] = useState<"inputs" | "outputs">("inputs") - // Configuration section: type-specific parts (code / prompt+model / agent) + selected part. - const [configParts, setConfigParts] = useState([]) - const [configLoading, setConfigLoading] = useState(false) - const [configPartKey, setConfigPartKey] = useState(null) - const setRefWorkflowId = useSetAtom(refWorkflowIdAtom) - - // Scope the revision picker to the selected workflow. - useEffect(() => { - setRefWorkflowId(selected?.id ?? null) - }, [selected?.id, setRefWorkflowId]) - - // Reset to a clean list state whenever the drawer is (re)opened. - useEffect(() => { - if (!open) return - setSearch("") - setFilter("all") - setSelected(null) - }, [open]) - - // Reset the axis selection when the chosen workflow changes, so a previous pick doesn't bleed in. - useEffect(() => { - setBindMode("revision") - setVersion(undefined) - setVariant(undefined) - setEnvironment(undefined) - setSchemaTab("inputs") - setConfigPartKey(null) - setDescription(selected?.description ?? "") - }, [selected?.id]) - - // Resolve the selected workflow's input + output schemas for the Schema section. - useEffect(() => { - if (!selected) { - setInputSchema(null) - setOutputSchema(null) - return - } - let cancelled = false - setSchemaLoading(true) - setInputSchema(null) - setOutputSchema(null) - Promise.all([ - bridge.resolveInputSchema(selected).catch(() => null), - bridge.resolveOutputSchema?.(selected).catch(() => null) ?? Promise.resolve(null), - ]) - .then(([input, output]) => { - if (cancelled) return - setInputSchema(input) - setOutputSchema(output) - }) - .finally(() => { - if (!cancelled) setSchemaLoading(false) - }) - return () => { - cancelled = true - } - }, [selected, bridge]) - - // Resolve the selected workflow's type-specific configuration for the Configuration section. - useEffect(() => { - if (!selected || !bridge.resolveConfigPayload) { - setConfigParts([]) - return - } - let cancelled = false - setConfigLoading(true) - setConfigParts([]) - bridge - .resolveConfigPayload(selected) - .then((payload) => { - if (!cancelled) setConfigParts(payload?.parts ?? []) - }) - .catch(() => { - if (!cancelled) setConfigParts([]) - }) - .finally(() => { - if (!cancelled) setConfigLoading(false) - }) - return () => { - cancelled = true - } - }, [selected, bridge]) - - const {environments, isLoading: environmentsLoading} = bridge.useWorkflowEnvironments(selected) - - // The picker selects a leaf id + carries the revision's version number in metadata. A leaf that - // equals the variant id means "the variant" (latest → no pinned version); else pin that version. - const handleRevisionSelect = (sel: WorkflowRevisionSelectionResult) => { - const isRevision = Boolean(sel.metadata.variantId) && sel.id !== sel.metadata.variantId - setVersion(isRevision ? String(sel.metadata.revision) : undefined) - // Keep the variant either way, so following its latest still identifies the variant. - setVariant(sel.metadata.variantId ? String(sel.metadata.variantId) : undefined) - } - - // List items carry no type (capability flags live on the revision URI), so the bridge resolves - // types by slug — plus a finer badge label for evaluators (their kind). Merge both in so badges, - // filter chips, and detail all read the real type. - const {typeBySlug, labelBySlug} = bridge.useWorkflowTypes(workflows) - const typedWorkflows = useMemo( - () => - workflows.map((w) => ({ - ...w, - type: typeBySlug[w.slug] ?? w.type, - typeLabel: labelBySlug?.[w.slug] ?? w.typeLabel, - })), - [workflows, typeBySlug, labelBySlug], - ) - - // Filter chips: only the types present in the available workflows (plus "All"). - const filterOptions = useMemo(() => { - const present = new Set(typedWorkflows.map((w) => w.type).filter(Boolean)) - const opts: {label: string; value: TypeFilter}[] = [{label: "All", value: "all"}] - for (const t of TYPE_FILTER_ORDER) { - if (present.has(t)) opts.push({label: capitalize(t), value: t}) - } - return opts - }, [typedWorkflows]) - - const filtered = useMemo(() => { - const q = search.trim().toLowerCase() - return typedWorkflows.filter((w) => { - if (filter !== "all" && w.type !== filter) return false - if (!q) return true - return `${w.slug} ${w.name ?? ""} ${w.description ?? ""}`.toLowerCase().includes(q) - }) - }, [typedWorkflows, search, filter]) - - const countProps = (schema: Record | null): number => - isRecord(schema?.properties) ? Object.keys(schema!.properties).length : 0 - - // Left-rail tabs for the Schema section. Outputs appears only when the bridge resolves an - // output schema with declared properties. - const schemaTabs = useMemo(() => { - const tabs: { - key: "inputs" | "outputs" - label: string - count: number - schema: Record | null - }[] = [ - {key: "inputs", label: "Inputs", count: countProps(inputSchema), schema: inputSchema}, - ] - if (countProps(outputSchema) > 0) { - tabs.push({ - key: "outputs", - label: "Outputs", - count: countProps(outputSchema), - schema: outputSchema, - }) - } - return tabs - }, [inputSchema, outputSchema]) - const activeSchema = - schemaTabs.find((t) => t.key === schemaTab)?.schema ?? schemaTabs[0]?.schema ?? null - - // Configuration section: the selected part (defaults to the first) and its content. - const activeConfigPart = - configParts.find((p) => p.key === configPartKey) ?? configParts[0] ?? null - - const canConfirm = Boolean(selected) && (bindMode === "revision" || Boolean(environment)) - - const handleConfirm = () => { - if (!selected) return - if (bindMode === "environment" && !environment) return - onSelect({ - slug: selected.slug, - refBy: bindMode === "revision" ? "variant" : "environment", - variant: bindMode === "revision" ? variant : undefined, - version: bindMode === "revision" ? version : undefined, - environment: bindMode === "environment" ? environment : undefined, - description: description.trim() || undefined, - }) - onClose() - } - - return ( - - - Reference a workflow -
- } - styles={{ - body: {padding: 0, display: "flex", flexDirection: "column", overflow: "hidden"}, - }} - > -
- {/* Master rail */} -
-
- - } - placeholder="Search workflows" - aria-label="Search workflows" - value={search} - onValueChange={setSearch} - allowClear - /> -

- The agent calls the chosen workflow as a tool; it runs server-side and - returns its output. -

- {filterOptions.length > 1 && ( -
- setFilter(val as TypeFilter)} - options={filterOptions} - aria-label="Filter workflows by type" - /> -
- )} -
- -
- {bridge.workflowsLoading ? ( -
- -
- ) : filtered.length === 0 ? ( - - No workflows to reference - - } - /> - ) : ( -
- {filtered.map((wf) => ( - - ))} -
- )} -
-
- - {/* Detail */} -
- {!selected ? ( -
-
- -
- Select a workflow -
-

- Preview its inputs and pick a version before adding it as a - tool. -

-
-
- ) : ( -
-
- {/* Header: workflow identity, above the sections */} -
- - - {selected.name || selected.slug} - - -
- - {/* Exposed-as + Description: root-level fields (no section chrome), - 2-panel to align with the sections' [rail | content] rhythm below. */} -
-
- Exposed as -
-
-
- {selected.slug} - -
-
-
- -
-
- Description -
-
- setDescription(e.target.value)} - autoSize={{minRows: 2, maxRows: 6}} - aria-label="Tool description" - placeholder="What this tool does and when the agent should call it" - /> -
-
- - } - title="Schema" - summary={`Inputs · ${countProps(inputSchema)}`} - summaryCollapsedOnly - > - {schemaLoading ? ( - - ) : ( - ({ - value: t.key, - label: t.label, - count: t.count, - }))} - value={schemaTab} - onChange={(v) => - setSchemaTab(v as "inputs" | "outputs") - } - > -
- -
- - )} - - - {(configLoading || configParts.length > 0) && ( - } - title="Configuration" - summary={ - configParts.length - ? `${configParts.length} ${ - configParts.length === 1 ? "part" : "parts" - }` - : undefined - } - summaryCollapsedOnly - > - {configLoading ? ( - - ) : ( - ({ - value: p.key, - label: p.label, - }))} - value={activeConfigPart?.key ?? ""} - onChange={setConfigPartKey} - > - {activeConfigPart ? ( - - ) : null} - - )} - - )} - - } - title="Reference by" - status={canConfirm ? "complete" : "warning"} - > - ({ - value: env.slug, - label: env.name || env.slug, - }))} - envLoading={environmentsLoading} - environmentSlug={environment} - onEnvironmentChange={setEnvironment} - envNotFound={ - environmentsLoading ? ( - - ) : ( - - No environments deployed - - ) - } - envHint="Calls whatever revision is deployed in the chosen environment." - /> - -
-
- )} -
-
- - - - ) -} diff --git a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/AddSubagentDrawer.tsx b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/AddSubagentDrawer.tsx new file mode 100644 index 0000000000..8b625fcd48 --- /dev/null +++ b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/AddSubagentDrawer.tsx @@ -0,0 +1,337 @@ +/** Pick the agents this agent can call. Presentational: every agent arrives as a prop. */ +import {useEffect, useMemo, useState} from "react" + +import {agentIconChrome, type AgentIconSelection} from "@agenta/ui/agent-icon" +import {LogoMarks} from "@agenta/ui/components/presentational" +import {EnhancedDrawer} from "@agenta/ui/drawer" +import {getProviderIcon} from "@agenta/ui/select-llm-provider" +import {cn} from "@agenta/ui/styles" +import {Button, EmptyState, SearchInput, SkeletonBlock} from "@agenta/ui/ui" +import {Check, Cube, Robot, Warning} from "@phosphor-icons/react" + +import {SubSectionHeader} from "../sectionGroups" + +import {CatalogListRow} from "./CatalogListRow" +import {INTEGRATION_DRAWER_WIDTH} from "./drawerWidths" +import {ExpandableDescription} from "./ExpandableDescription" + +/** One connected app on an agent. */ +export interface SubagentIntegration { + /** Integration key, e.g. "github". Doubles as the React key and the fallback label. */ + key: string + name?: string + logo?: string | null +} + +/** One selectable agent. */ +export interface SubagentOption { + /** The agent's identity, and the key the caller adds and removes by. */ + id: string + name: string + description?: string + /** The agent's chosen icon. Falls back to a robot glyph when the author never picked one. */ + icon?: AgentIconSelection | null + /** The model this agent runs on, e.g. "claude-sonnet-4-5". */ + model?: string + /** Provider display name, e.g. "Anthropic". An unknown name draws a neutral glyph. */ + provider?: string + integrations?: SubagentIntegration[] + /** Already a subagent of the agent being edited. Its action removes instead of adding. */ + added?: boolean +} + +export interface AddSubagentDrawerProps { + open: boolean + onClose: () => void + /** Every agent in the project, minus the one being edited. */ + options: SubagentOption[] + loading?: boolean + /** How many agents could not be loaded, so the list can say so instead of hiding them. */ + failedCount?: number + onRetry?: () => void + /** One write per author action. May be async; the drawer disables its actions until it settles. */ + onAdd: (options: SubagentOption[]) => void | Promise + onRemove: (options: SubagentOption[]) => void | Promise +} + +const ICON_BOX = "flex size-7 items-center justify-center rounded-md" + +/** The model an agent runs on, marked with its provider's logo. */ +function ModelChip({model, provider}: {model: string; provider?: string}) { + const ProviderIcon = provider ? getProviderIcon(provider) : null + return ( + + + {ProviderIcon ? : } + + {model} + + ) +} + +/** Separates the model from the connected apps. A gap alone let the two runs read as one list. */ +const MetaDot = () => ( + +) + +function SubagentRow({ + option, + busy, + onAdd, + onRemove, +}: { + option: SubagentOption + busy?: boolean + onAdd: () => void + onRemove: () => void +}) { + const [expanded, setExpanded] = useState(false) + const chrome = agentIconChrome(option.icon, { + size: 14, + fallbackGlyph: , + fallbackClassName: "bg-[var(--ag-colorFillSecondary)] text-[var(--ag-colorTextSecondary)]", + }) + const integrations = option.integrations ?? [] + + return ( + + {chrome.glyph} + + } + title={option.name} + titleSuffix={ + option.added ? ( + + + Added + + ) : null + } + action={ + option.added ? ( + + ) : ( + + ) + } + > + + + {option.model ? ( + + ) : null} + {option.model ? : null} + + No connected apps + + } + /> + + + ) +} + +/** A loading row. SkeletonBlock, never Skeleton: the latter is the antd composite and draws four + * overlapping bars in a box meant for one. CatalogListRow truncates its title, which collapses a bar. */ +function RowSkeleton({widths}: {widths: [string, string]}) { + return ( +
+ +
+
+ +
+ + +
+ +
+ ) +} + +/** Uneven widths: three identical bars read as a loading graphic, not as rows about to arrive. */ +const SKELETON_WIDTHS: [string, string][] = [ + ["w-36", "w-full"], + ["w-28", "w-4/5"], + ["w-44", "w-3/5"], +] + +export function AddSubagentDrawer({ + open, + onClose, + options, + loading, + failedCount = 0, + onRetry, + onAdd, + onRemove, +}: AddSubagentDrawerProps) { + const [search, setSearch] = useState("") + // One write in flight at a time: two overlapping ones both start from the same array. + const [busy, setBusy] = useState(false) + const run = async (write: () => void | Promise) => { + if (busy) return + setBusy(true) + try { + await write() + } finally { + setBusy(false) + } + } + + // Reset on the `open` transition: `destroyOnClose` unmounts the body, not this component. + useEffect(() => { + if (!open) setSearch("") + }, [open]) + + const visible = useMemo(() => { + const q = search.trim().toLowerCase() + if (!q) return options + return options.filter( + (o) => + o.name.toLowerCase().includes(q) || (o.description ?? "").toLowerCase().includes(q), + ) + }, [options, search]) + + // Add all acts on what the search shows, never on hidden rows. + const addable = useMemo(() => visible.filter((o) => !o.added), [visible]) + + const handleClose = () => { + setSearch("") + onClose() + } + + return ( + +
+ + Add subagents +
+ + Pick the agents this agent can call. + +
+ } + styles={{ + body: {padding: 0, display: "flex", flexDirection: "column", overflow: "hidden"}, + }} + // Each row adds itself, so the only thing left for the footer is to close. + footer={ +
+ +
+ } + > +
+ + + {failedCount > 0 ? ( +
+ + + {failedCount} {failedCount === 1 ? "agent" : "agents"} could not be + loaded, so {failedCount === 1 ? "it is" : "they are"} not listed. + + {onRetry ? ( + + ) : null} +
+ ) : null} + + {loading ? ( +
+ {SKELETON_WIDTHS.map((widths, index) => ( + + ))} +
+ ) : visible.length === 0 ? ( + + ) : ( +
+ 1 ? ( + + ) : undefined + } + /> +
+ {visible.map((option) => ( + void run(() => onAdd([option]))} + onRemove={() => void run(() => onRemove([option]))} + /> + ))} +
+
+ )} +
+ + ) +} diff --git a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/AgentIntegrationDrawer.tsx b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/AgentIntegrationDrawer.tsx index fb812c1473..0a65b389e8 100644 --- a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/AgentIntegrationDrawer.tsx +++ b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/AgentIntegrationDrawer.tsx @@ -37,7 +37,9 @@ import ConnectDrawer from "../../../gatewayTool/drawers/ConnectDrawer" import {ProviderLogo, SubSectionHeader} from "../sectionGroups" import type {GatewayConnectionTarget, IntegrationRow} from "../toolUtils" +import {CatalogListRow} from "./CatalogListRow" import {INTEGRATION_DRAWER_WIDTH} from "./drawerWidths" +import {ExpandableDescription} from "./ExpandableDescription" import {catalogSections, type CategorySelection} from "./integrationCatalogFilters" type CatalogIntegration = ToolCatalogIntegration | ToolCatalogIntegrationDetails @@ -127,36 +129,28 @@ function ConnectedRow({ const chooserButton = multiple && (!added || swappable) return ( -
-
- -
- {name} - - {subtitle} - -
- {added && !choosing ? ( - + } + title={name} + titleSuffix={ + added && !choosing ? ( + Added - ) : null} - {!added && !multiple && !isConnectionValid(single) ? ( - + ) : !added && !multiple && !isConnectionValid(single) ? ( + needs reconnect - ) : null} - {chooserButton ? ( + ) : null + } + action={ + chooserButton ? ( - ) : null} - {!chooserButton && !added ? ( + ) : !added ? ( - ) : null} -
- {choosing ? ( -
- - {group.connections.map((connection) => ( -
+ ) : null + } + > + {subtitle} + ) } @@ -219,18 +217,20 @@ function CatalogRow({ onConnect: () => void }) { return ( -
- -
- {integration.name} - - {integration.description} - -
- -
+ } + title={integration.name} + action={ + + } + > + + ) } @@ -421,7 +421,7 @@ function IntegrationCatalogContent({ label="Connected in your workspace" count={connectedGroups.length} /> -
+
{connectedGroups.map((group) => ( ) : ( -
+
{catalogRows.map((integration) => ( void -} - -// A fresh tool-definition seed. The schema editor opens on this and only appends on Save, so a -// half-filled tool never lands in the config (mirrors the legacy custom-tool create flow). -function buildToolDefinitionSeed(): ToolObj { - return { - type: "function", - function: { - name: "get_weather", - description: "Get current weather", - parameters: { - type: "object", - properties: {location: {type: "string", description: "City name"}}, - required: ["location"], - additionalProperties: false, - }, - }, - } -} - -export const AgentToolSelectorPopover = memo(function AgentToolSelectorPopover({ - onAddTool, - disabled = false, - gatewayTools: gatewayToolsProp, - trigger, - onReferenceWorkflow, - onOpenIntegration, -}: AgentToolSelectorPopoverProps) { - const {gatewayTools: gatewayToolsFromContext, workflowReference} = useDrillInUI() - const gatewayTools = gatewayToolsProp ?? gatewayToolsFromContext - - const showReference = Boolean(workflowReference?.enabled && onReferenceWorkflow) - const showIntegration = Boolean(gatewayTools?.enabled) - - const groups: AddItemGroup[] = [] - - const addExisting: AddItemGroup["items"] = [] - if (showReference) { - addExisting.push({ - key: "reference", - icon: , - title: "Reference a workflow", - subtitle: "Call a published workflow as a tool", - opensDrawer: true, - onSelect: onReferenceWorkflow, - }) - } - if (showIntegration) { - addExisting.push({ - key: "integration", - icon: , - title: "Third-party integration", - subtitle: "Connect an app, pick actions", - opensDrawer: true, - onSelect: () => - onOpenIntegration ? onOpenIntegration() : gatewayTools?.onOpenCatalog(), - }) - } - if (addExisting.length) groups.push({label: "Add existing", items: addExisting}) - - groups.push({ - label: "Create new", - items: [ - { - key: "definition", - icon: , - title: "Tool definition", - subtitle: "JSON schema, executed by your app", - // The schema editor opens on this seed and only appends on Save. - onSelect: () => onAddTool(buildToolDefinitionSeed(), {source: "custom"}), - }, - { - key: "ai", - icon: , - title: "Create with AI", - subtitle: "Describe a tool and let AI build it", - disabled: true, - disabledHint: "Coming soon", - }, - ], - }) - - return ( - - - Tool - - ) - } - /> - ) -}) diff --git a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/CatalogListRow.tsx b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/CatalogListRow.tsx new file mode 100644 index 0000000000..ad7c416d10 --- /dev/null +++ b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/CatalogListRow.tsx @@ -0,0 +1,51 @@ +/** One agent-picker row: a leading mark, a title with a meta line, and an action on the right. */ +import type {ReactNode} from "react" + +export interface CatalogListRowProps { + /** Logo, icon chip, or anything else that identifies the item. */ + leading?: ReactNode + title: ReactNode + /** Tags and markers on the title line, after the name. */ + titleSuffix?: ReactNode + /** Description, meta line, or both. Sits under the title. */ + children?: ReactNode + /** The row's action, right-aligned and vertically centred with the title line. */ + action?: ReactNode + /** Tints the row, for the open state of an expandable description. */ + highlighted?: boolean + /** Expanded rows below the title, such as a connection chooser. */ + expansion?: ReactNode +} + +export function CatalogListRow({ + leading, + title, + titleSuffix, + children, + action, + highlighted, + expansion, +}: CatalogListRowProps) { + return ( +
+ {/* items-start, not items-center: the action must stay put while the row grows. */} +
+ {leading ? {leading} : null} +
+ {/* min-h matches a small Button, so the title line stays level with the action. */} +
+ {title} + {titleSuffix} +
+ {children} +
+ {action ? {action} : null} +
+ {expansion} +
+ ) +} diff --git a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/ExpandableDescription.tsx b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/ExpandableDescription.tsx new file mode 100644 index 0000000000..a72992ced8 --- /dev/null +++ b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/ExpandableDescription.tsx @@ -0,0 +1,98 @@ +/** A description that clamps, offering Show more only when clamping really hid something. */ +import {useId, useLayoutEffect, useState} from "react" + +import {isDescriptionTruncatable} from "../integrationPolicy" + +/** Literal class names: Tailwind never generates a `line-clamp-${n}` built at runtime. */ +const CLAMP: Record = { + 1: "truncate", + 2: "line-clamp-2", +} + +const TEXT_CLASS = "text-xs text-[var(--ag-colorTextTertiary)]" +const EXPANDED_CLASS = "whitespace-pre-line leading-relaxed text-[var(--ag-colorTextSecondary)]" + +export interface ExpandableDescriptionProps { + description?: string + /** Lines to clamp to while collapsed. 1 truncates on width; 2 or more clamp on height. */ + lines?: number + /** Told whether the description is open, for a parent that restyles its own row. */ + onExpandedChange?: (expanded: boolean) => void + /** Names what the toggle expands, so a list of Show more buttons stays distinguishable. */ + label?: string +} + +export function ExpandableDescription({ + description, + lines = 1, + onExpandedChange, + label, +}: ExpandableDescriptionProps) { + const textId = useId() + const [expanded, setExpandedState] = useState(false) + const [preview, setPreview] = useState(null) + const [overflows, setOverflows] = useState(false) + const text = description?.trim() + + // Measured while collapsed, on the axis the clamp acts on: width for 1 line, height for 2+. + useLayoutEffect(() => { + if (!text) { + setOverflows(false) + return + } + if (!preview || expanded) return + const measure = () => + setOverflows( + lines > 1 + ? preview.scrollHeight > preview.clientHeight + 1 + : preview.scrollWidth > preview.clientWidth, + ) + measure() + // Resize-watch only a multi-line clamp: single-line lists are the long ones, and one + // observer per row there costs more than the re-measure it would catch. + if (lines < 2 || typeof ResizeObserver === "undefined") return + const observer = new ResizeObserver(measure) + observer.observe(preview) + return () => observer.disconnect() + }, [preview, expanded, text, lines]) + + const setExpanded = (next: boolean) => { + setExpandedState(next) + onExpandedChange?.(next) + } + + if (!text) return null + + const truncatable = isDescriptionTruncatable(text, overflows) + const clamp = CLAMP[lines] ?? CLAMP[1] + + return ( + <> + + {text} + + {truncatable ? ( + + ) : null} + + ) +} diff --git a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/IntegrationPermissionDrawer.tsx b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/IntegrationPermissionDrawer.tsx index 1b27d33b61..8979dfd5c9 100644 --- a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/IntegrationPermissionDrawer.tsx +++ b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/IntegrationPermissionDrawer.tsx @@ -13,7 +13,7 @@ * Built for scale: a provider integration can list 50 to 200 tools, so the body carries a search * box, two collapsible groups (read-only and write and delete), and a per-group row cap. */ -import {memo, useLayoutEffect, useMemo, useState} from "react" +import {memo, useMemo, useState} from "react" import { useToolConnectionsQuery, @@ -33,7 +33,6 @@ import ConnectionStatusBadge from "../../../gatewayTool/components/ConnectionSta import { INTEGRATION_PRESETS, TOOL_PERMISSION_OPTIONS, - isDescriptionTruncatable, partitionToolsByAccess, presetPermissions, readIntegrationPreset, @@ -58,6 +57,7 @@ import type { } from "../toolUtils" import {INTEGRATION_DRAWER_WIDTH} from "./drawerWidths" +import {ExpandableDescription} from "./ExpandableDescription" import {humanizeActionKey} from "./itemDescriptors" import {PolicyGlyph} from "./PermissionGlyph" import {PermissionPolicySelect} from "./PermissionPolicySelect" @@ -110,20 +110,8 @@ const ToolRow = memo(function ToolRow({ onChange: (toolKey: string, permission: GatewayPermission) => void disabled?: boolean }) { + // Only the row's tint depends on this; the clamp and the toggle live in ExpandableDescription. const [expanded, setExpanded] = useState(false) - const [preview, setPreview] = useState(null) - const [overflows, setOverflows] = useState(false) - const description = tool.description?.trim() - // Measured only while collapsed: expanding changes the very box the measurement reads. - useLayoutEffect(() => { - if (!description) { - setOverflows(false) - return - } - if (!preview || expanded) return - setOverflows(preview.scrollWidth > preview.clientWidth) - }, [preview, expanded, description]) - const truncatable = isDescriptionTruncatable(description, overflows) return (
) : null}
- {description ? ( - - {description} - - ) : null} - {truncatable ? ( - - ) : null} +
{descriptor.icon ?? descriptor.mono} @@ -119,10 +124,10 @@ export function ItemRow({ // The whole row opens it; the chevron and tags used to be a dead target. onClick={interactive ? onEdit : undefined} className={cn( - "group flex items-center gap-2.5 rounded border border-solid border-[var(--ag-c-EAEFF5)] px-3 py-2 transition-colors", + "group flex items-center gap-2.5 rounded-lg border border-solid border-[var(--ag-colorBorderSecondary)] py-2.5 pl-3 pr-2 transition-colors", // Item cards read as white sheets sitting ON the expanded section's band. !locked && "bg-[var(--ag-surface-section-content)]", - interactive && !status && "cursor-pointer hover:border-[var(--ag-zinc-5)]", + interactive && !status && "cursor-pointer hover:bg-[var(--ag-colorFillQuaternary)]", interactive && status && "cursor-pointer", locked && "bg-[var(--ant-color-fill-quaternary)] opacity-70", )} @@ -151,14 +156,14 @@ export function ItemRow({
{descriptor.name}
{descriptor.description ? ( - + {descriptor.description} ) : null} @@ -181,12 +186,15 @@ export function ItemRow({ e.stopPropagation() onRemove() }} - className="flex cursor-pointer items-center border-0 bg-transparent p-0 text-[var(--ag-zinc-5)] opacity-0 transition-opacity hover:text-colorError group-hover:opacity-100" + // A 24px ghost target, not a bare glyph: a hover-only icon is hard to hit. + className="flex size-6 cursor-pointer items-center justify-center rounded border-0 bg-transparent p-0 text-[var(--ag-colorTextTertiary)] opacity-0 transition-opacity hover:bg-[var(--ag-colorErrorBg)] hover:text-[var(--ag-colorErrorText)] focus-visible:opacity-100 focus-visible:outline focus-visible:outline-2 focus-visible:outline-offset-1 focus-visible:outline-[var(--ag-colorPrimary)] group-hover:opacity-100" > ) : null} - {interactive ? : null} + {interactive ? ( + + ) : null}
) diff --git a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/SubagentDrawerContainer.tsx b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/SubagentDrawerContainer.tsx new file mode 100644 index 0000000000..0c70f1674c --- /dev/null +++ b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/SubagentDrawerContainer.tsx @@ -0,0 +1,196 @@ +/** The two connected halves of the Subagents section: the picker and the saved list. */ +import {useCallback, useMemo} from "react" + +import {agentIconAtomFamily, workflowMolecule} from "@agenta/entities/workflow" +import {agentIconChrome} from "@agenta/ui/agent-icon" +import type { + WorkflowReferenceBridge, + WorkflowReferencePayload, + WorkflowReferenceUI, +} from "@agenta/ui/drill-in" +import {Robot} from "@phosphor-icons/react" +import {useAtomValue} from "jotai" + +import {useFamilyMap} from "../hooks/useFamilyMap" +import {useIntegrationLogos} from "../hooks/useIntegrationLogos" + +import {AddSubagentDrawer, type SubagentOption} from "./AddSubagentDrawer" +import {toolReferenceSlug} from "./itemDescriptors" +import {SubagentList, type SubagentListProps} from "./ToolManagementList" + +const iconFamily = (id: string) => agentIconAtomFamily(id) + +export interface SubagentDrawerContainerProps { + open: boolean + onClose: () => void + bridge: WorkflowReferenceBridge + /** The revision being edited, so the agent cannot be offered itself. */ + revisionId: string | null + /** Slugs already saved as references on this agent, agent-typed or not. */ + savedSlugs: string[] + onAdd: (payload: WorkflowReferencePayload) => Promise + onRemoveSlug: (slug: string) => void +} + +export function SubagentDrawerContainer({ + open, + onClose, + bridge, + revisionId, + savedSlugs, + onAdd, + onRemoveSlug, +}: SubagentDrawerContainerProps) { + // Gated on `open` throughout: the container outlives the drawer and must idle when closed. + const projectSlugs = useMemo( + () => (open ? bridge.workflows.map((w) => w.slug).filter(Boolean) : []), + [open, bridge.workflows], + ) + const { + bySlug, + failedSlugs, + loading: catalogLoading, + retry, + } = bridge.useWorkflowReferenceCatalog(projectSlugs) + + // The agent being edited, matched on its WORKFLOW id. Offering it to itself loops the runner. + const selfWorkflowId = useAtomValue(workflowMolecule.selectors.workflowId(revisionId ?? "")) as + | string + | undefined + + // Agents only. Every type is undefined while the catalog loads, so the drawer shows loading. + const agents = useMemo( + () => + open + ? bridge.workflows.filter((w) => { + if (!w.slug || (selfWorkflowId && w.id === selfWorkflowId)) return false + return bySlug[w.slug]?.type === "agent" + }) + : [], + [open, bridge.workflows, bySlug, selfWorkflowId], + ) + + // Icons for the listed agents only: the whole project list leaves permanent family entries. + const idsKey = useMemo( + () => + agents + .map((w) => w.id) + .filter(Boolean) + .join("\n"), + [agents], + ) + const iconById = useFamilyMap(idsKey, iconFamily) + + // Logos for every app any listed agent connects, resolved once for the batch. + const logoKeys = useMemo( + () => agents.flatMap((wf) => bySlug[wf.slug]?.integrations ?? []), + [agents, bySlug], + ) + const logoByKey = useIntegrationLogos(logoKeys) + + const options = useMemo(() => { + const saved = new Set(savedSlugs) + return agents.map((wf: WorkflowReferenceUI) => { + const entry = bySlug[wf.slug] + return { + id: wf.slug, + name: wf.name || wf.slug, + description: wf.description, + icon: iconById.get(wf.id) ?? null, + model: entry?.model, + provider: entry?.provider, + integrations: (entry?.integrations ?? []).map( + (key) => logoByKey.get(key) ?? {key, name: key, logo: null}, + ), + added: saved.has(wf.slug), + } + }) + }, [agents, bySlug, iconById, logoByKey, savedSlugs]) + + // Sequential on purpose: each add re-reads the freshest config after an await. + const handleAdd = useCallback( + async (selected: SubagentOption[]) => { + // No pinned version: the server reads a bare variant slug as the latest revision. + for (const option of selected) { + await onAdd({slug: option.id}) + } + }, + [bySlug, onAdd], + ) + + const handleRemove = useCallback( + (selected: SubagentOption[]) => { + for (const option of selected) onRemoveSlug(option.id) + }, + [onRemoveSlug], + ) + + return ( + + ) +} + +/** The Subagents section body. Resolves only the saved slugs: the project-wide workflow list + * stays empty until the picker is opened once. */ +export function ConnectedSubagentList({ + bridge, + ...listProps +}: SubagentListProps & {bridge: WorkflowReferenceBridge}) { + const savedSlugs = useMemo( + () => + listProps.entries + .map(({item}) => toolReferenceSlug(item)) + .filter((s): s is string => Boolean(s)), + [listProps.entries], + ) + const {bySlug} = bridge.useWorkflowReferenceCatalog(savedSlugs) + + // Each saved subagent's icon, from the agent it points at. The list stays presentational. + const iconIdsKey = useMemo( + () => + savedSlugs + .map((slug) => bySlug[slug]?.workflowId) + .filter((id): id is string => Boolean(id)) + .join("\n"), + [savedSlugs, bySlug], + ) + const iconById = useFamilyMap(iconIdsKey, iconFamily) + const chromeBySlug = useMemo(() => { + const map = new Map< + string, + {glyph: React.ReactNode; className: string; style?: React.CSSProperties} + >() + for (const slug of savedSlugs) { + const workflowId = bySlug[slug]?.workflowId + const chrome = agentIconChrome(workflowId ? (iconById.get(workflowId) ?? null) : null, { + size: 15, + fallbackGlyph: , + fallbackClassName: + "bg-[var(--ag-colorFillSecondary)] text-[var(--ag-colorTextSecondary)]", + }) + map.set(slug, {glyph: chrome.glyph, className: chrome.className, style: chrome.style}) + } + return map + }, [savedSlugs, bySlug, iconById]) + + // Marks only what the catalog resolved, so a slow fetch never mislabels a good agent. + const nonAgentSlugs = useMemo(() => { + const slugs = new Set() + for (const slug of savedSlugs) { + const type = bySlug[slug]?.type + if (type && type !== "agent") slugs.add(slug) + } + return slugs + }, [savedSlugs, bySlug]) + + return +} diff --git a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/SubagentHeader.tsx b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/SubagentHeader.tsx new file mode 100644 index 0000000000..0bdcfd3f34 --- /dev/null +++ b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/SubagentHeader.tsx @@ -0,0 +1,53 @@ +/** The subagent drawer header's two resolved parts. Both take the bridge as a required prop and + * mount only when the host supplies one, so neither calls a hook conditionally. */ +import {agentIconAtomFamily} from "@agenta/entities/workflow" +import {agentIconChrome} from "@agenta/ui/agent-icon" +import type {WorkflowReferenceBridge} from "@agenta/ui/drill-in" +import {cn} from "@agenta/ui/styles" +import {Button} from "@agenta/ui/ui" +import {Robot} from "@phosphor-icons/react" +import {useAtomValue} from "jotai" + +export function SubagentHeaderIcon({ + bridge, + slug, +}: { + bridge: WorkflowReferenceBridge + slug: string +}) { + const {detail} = bridge.useSubagentDetail(slug) + const record = useAtomValue(agentIconAtomFamily(detail?.workflowId ?? "")) + const chrome = agentIconChrome(record, { + size: 16, + fallbackGlyph: , + fallbackClassName: "bg-[var(--ag-colorFillSecondary)] text-[var(--ag-colorTextSecondary)]", + }) + return ( + + {chrome.glyph} + + ) +} + +export function SubagentOpenAgentButton({ + bridge, + slug, +}: { + bridge: WorkflowReferenceBridge + slug: string +}) { + const {detail} = bridge.useSubagentDetail(slug) + const href = detail?.workflowId ? (bridge.agentHref?.(detail.workflowId) ?? null) : null + if (!href) return null + return ( + + ) +} diff --git a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/ToolManagementList.tsx b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/ToolManagementList.tsx index 4e0783597b..8cdfa404de 100644 --- a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/ToolManagementList.tsx +++ b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/ToolManagementList.tsx @@ -1,32 +1,23 @@ -/** - * ToolManagementList - * - * The Tools section body. Connected apps are listed as INTEGRATIONS: one row per integration, - * whatever format its entries are saved in. Adding an integration adds all of its tools, so the row - * summarizes the integration's permission policy and opens the permission drawer; there is no - * per-row expansion and no per-row plus any more. The other kinds — workflow references, tool - * definitions, and built-ins — stay flat row lists. - * - * A row is built from BOTH saved formats. An integration still on the legacy per-action entries - * carries an "old format" tag until its drawer is opened, which is what migrates it. - * - * Provider details (for app names and logos) only load when integrations exist — each row's detail - * hook mounts only when that row exists. Dark-safe (`--ag-color*` tokens only). - */ -import {type ReactNode, useMemo} from "react" +/** The Integrations and Subagents section bodies, both read off the flat `tools` array. Function + * tool definitions and provider built-ins are deliberately not rendered: they still run. */ +import {type CSSProperties, type ReactNode, useMemo} from "react" import {useToolIntegrationDetail} from "@agenta/entities/gatewayTool" import type {ConfigItemView} from "../ConfigItemDrawer" import {integrationPermissionSummary} from "../integrationPolicy" -import {ProviderLogo, SubSectionHeader} from "../sectionGroups" +import {ProviderLogo} from "../sectionGroups" import {integrationRowIndices, isHarnessBuiltinTool, type IntegrationRow} from "../toolUtils" -import {describeTool, isFunctionTool} from "./itemDescriptors" +import { + describeSubagent, + isReferenceTool, + toolReferenceSlug, + type ItemDescriptor, +} from "./itemDescriptors" import {ITEM_KINDS} from "./itemKinds" import {ItemRow, type ItemRowStatus, type ItemRowStatusTone} from "./ItemRow" import {PolicyGlyph} from "./PermissionGlyph" -import {SectionAddButton} from "./SectionAddButton" /** Per-tool draft/validation status, keyed by the tool's index in the flat `tools` array. */ type ToolStatusFor = (item: unknown, index: number) => ItemRowStatus | undefined @@ -139,99 +130,47 @@ function IntegrationListRow({ export interface ToolManagementListProps { tools: unknown[] - /** The integration rows, derived from the same `tools` by the owning hook. Passed in rather - * than rebuilt here, so the rows this list renders ARE the ones the drawers act on. */ + /** The integration rows, passed in so the rows rendered ARE the ones the drawers act on. */ integrationRows: IntegrationRow[] - openEdit: (kind: "tool", index: number, item: unknown, view: ConfigItemView) => void - removeItem: (kind: "tool", index: number) => void - closeEditor: () => void disabled?: boolean - /** Opens the add-integration drawer (the integrations header plus). */ - onAddIntegration?: () => void /** Opens one integration's permission drawer. Migrates its legacy entries first. */ onOpenIntegration?: (row: IntegrationRow) => void /** Drops every entry an integration owns, in one write. */ onRemoveIntegration?: (row: IntegrationRow) => void - /** Add trigger shown in the empty state (the tool selector popover). */ - emptyAdd: ReactNode + /** Add trigger shown in the empty state. Omitted when there is no drawer to open. */ + emptyAdd?: ReactNode /** Per-tool draft/validation status (unsaved edits, missing fields). */ statusFor?: ToolStatusFor } -/** A flat, headed sub-section of bordered item rows (references / definitions / built-in). */ -function FlatToolSection({ - label, - entries, - openEdit, - removeItem, - closeEditor, - disabled, - statusFor, -}: { - label: string - entries: IndexedTool[] - openEdit: ToolManagementListProps["openEdit"] - removeItem: ToolManagementListProps["removeItem"] - closeEditor: () => void - disabled?: boolean - statusFor?: ToolStatusFor -}) { - if (entries.length === 0) return null +/** Shared empty-state line. The add half is optional: a host with no drawer renders no control. */ +function EmptyLine({label, add}: {label: string; add?: ReactNode}) { return ( -
- -
- {entries.map(({item, index}) => ( - openEdit("tool", index, item, ITEM_KINDS.tool.editView(item))} - onRemove={() => { - removeItem("tool", index) - closeEditor() - }} - disabled={disabled || ITEM_KINDS.tool.isReadOnly(item)} - status={statusFor?.(item, index)} - /> - ))} -
-
+ + {label} + {add ? <> — {add} : null} + ) } -/** - * The integration rows. Isolated in its own component so the (paginated) catalog detail queries - * only run when integrations actually exist. - */ -function IntegrationSection({ - rows, +/** The Integrations section body: one row per connected app, no sub-header. */ +export function ToolManagementList({ tools, + integrationRows, disabled, - onAddIntegration, onOpenIntegration, onRemoveIntegration, + emptyAdd, statusFor, -}: { - rows: IntegrationRow[] - tools: unknown[] - disabled?: boolean - onAddIntegration?: () => void - onOpenIntegration?: (row: IntegrationRow) => void - onRemoveIntegration?: (row: IntegrationRow) => void - statusFor?: ToolStatusFor -}) { +}: ToolManagementListProps) { + if (integrationRows.length === 0) { + if (disabled) return null + return + } + return (
- - ) : undefined - } - /> - {rows.map((row) => ( + {integrationRows.map((row) => ( { + // Legacy harness built-ins are inert: they render nowhere. + if (isHarnessBuiltinTool(item) || claimed.has(index)) return + if (isReferenceTool(item)) entries.push({item, index}) + }) + return entries +} + +export interface SubagentListProps { + entries: IndexedTool[] + /** Saved references whose workflow is not an agent. Listed and removable, never addable. */ + nonAgentSlugs?: Set + /** Each subagent's icon chrome by slug. Only the caller can reach the icon record. */ + chromeBySlug?: Map + openEdit: (kind: "tool", index: number, item: unknown, view: ConfigItemView) => void + removeItem: (kind: "tool", index: number) => void + closeEditor: () => void + disabled?: boolean + /** Add trigger shown in the empty state. Omitted when there is no picker to open. */ + emptyAdd?: ReactNode + statusFor?: ToolStatusFor +} + +/** The Subagents section body: a flat row list, no sub-header. */ +/** Tag a reference whose workflow is not an agent, so it stays visible and removable. */ +function markNonAgent( + descriptor: ItemDescriptor, + item: unknown, + nonAgentSlugs?: Set, +): ItemDescriptor { + if (!nonAgentSlugs?.size) return descriptor + const slug = toolReferenceSlug(item) + if (!slug || !nonAgentSlugs.has(slug)) return descriptor + return {...descriptor, tags: [...(descriptor.tags ?? []), "not an agent"]} +} + +/** Stable per-entry key: the saved reference's own identity, never its array position. */ +function subagentKey(item: unknown, index: number): string { + const t = (item ?? {}) as Record + const name = typeof t.name === "string" ? t.name : "" + const identity = [toolReferenceSlug(item) ?? "", name].filter(Boolean).join("|") + // A reference with no identity falls back to its position, rather than colliding. + return identity || `subagent-${index}` +} + +export function SubagentList({ + entries, + nonAgentSlugs, + chromeBySlug, openEdit, removeItem, closeEditor, disabled, - onAddIntegration, - onOpenIntegration, - onRemoveIntegration, emptyAdd, statusFor, -}: ToolManagementListProps) { - // Partition the rest by kind, preserving each tool's original index (edit and remove address - // the flat array). The integration rows already claim their own positions. - const {references, definitions, builtins, visibleCount} = useMemo(() => { - const references: IndexedTool[] = [] - const definitions: IndexedTool[] = [] - const builtins: IndexedTool[] = [] - const claimed = new Set(integrationRows.flatMap(integrationRowIndices)) - tools.forEach((item, index) => { - // Legacy harness built-ins are inert: they render nowhere. - if (isHarnessBuiltinTool(item) || claimed.has(index)) return - const t = (item ?? {}) as Record - if (t.type === "reference") { - references.push({item, index}) - return - } - if (!isFunctionTool(item)) { - builtins.push({item, index}) - return - } - definitions.push({item, index}) - }) - const visibleCount = claimed.size + references.length + definitions.length + builtins.length - return {references, definitions, builtins, visibleCount} - }, [tools, integrationRows]) - - // A config carrying only legacy built-in entries renders as empty, so it gets the empty state. - if (visibleCount === 0) { +}: SubagentListProps) { + if (entries.length === 0) { if (disabled) return null - return ( - - {ITEM_KINDS.tool.emptyLabel} — {emptyAdd} - - ) + return } return ( -
- {integrationRows.length > 0 && ( - + {entries.map(({item, index}) => ( + openEdit("tool", index, item, ITEM_KINDS.tool.editView(item))} + onRemove={() => { + removeItem("tool", index) + closeEditor() + }} + disabled={disabled || ITEM_KINDS.tool.isReadOnly(item)} + status={statusFor?.(item, index)} /> - )} - - - + ))}
) } diff --git a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/itemDescriptors.tsx b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/itemDescriptors.tsx index a2d6ede9b1..5f2777bd7a 100644 --- a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/itemDescriptors.tsx +++ b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/itemDescriptors.tsx @@ -3,7 +3,7 @@ * instructions file) to an {@link ItemDescriptor} (avatar, name, description, tags). Kept beside the * predicates they rely on (`isFunctionTool`, `isStaticSkill`) so registry, rows, and drawers agree. */ -import {FileText, GraphIcon, Plugs} from "@phosphor-icons/react" +import {FileText, GraphIcon, Plugs, Robot} from "@phosphor-icons/react" import {parseGatewayEntry, type ToolObj} from "../toolUtils" @@ -21,6 +21,10 @@ export interface ItemDescriptor { color: string /** Avatar icon (overrides the monogram). */ icon?: React.ReactNode + /** Avatar chip classes, for an item that paints its own chip. Set with `avatarStyle`. */ + avatarClassName?: string + /** Custom properties the chip classes read (the light and dark tint and ink). */ + avatarStyle?: React.CSSProperties /** Type tags shown on the right of a row (e.g. "built-in", "definition", "gmail"). */ tags: string[] /** Type label for the drawer header badge (e.g. "definition", "MCP server"). */ @@ -139,6 +143,36 @@ export function humanizeActionKey(key: string): string { .join(" ") } +/** A saved subagent row. Not `describeTool`: that returns the internal vocabulary (a "workflow" + * tag, a teal square, a monospace name), none of which is true of another agent. */ +export function describeSubagent( + tool: unknown, + chrome?: {glyph: React.ReactNode; className: string; style?: React.CSSProperties}, +): ItemDescriptor { + const t = (tool ?? {}) as Record + const slug = typeof t.slug === "string" ? t.slug : undefined + const name = typeof t.name === "string" && t.name ? t.name : slug + return { + name: name ?? "Subagent", + // Prose, never monospace: this is an agent's name, not an identifier. + monoName: false, + description: typeof t.description === "string" ? t.description : undefined, + mono: "", + color: "transparent", + icon: chrome?.glyph ?? , + // Always chipped: an unchipped avatar paints white on transparent and the glyph vanishes. + avatarClassName: + chrome?.className ?? + "bg-[var(--ag-colorFillSecondary)] text-[var(--ag-colorTextSecondary)]", + avatarStyle: chrome?.style, + // No type tag. "workflow" is an internal type and nothing user-meaningful replaces it. + tags: [], + typeLabel: "subagent", + typeColor: "geekblue", + subtitle: slug ? `Subagent · ${slug}` : "Subagent", + } +} + /** Classify a tool into its row avatar / name / description / type tags. */ export function describeTool(tool: unknown): ItemDescriptor { const t = (tool ?? {}) as Record diff --git a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/itemKinds.tsx b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/itemKinds.tsx index 285f6ddce5..fb6cb97e85 100644 --- a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/itemKinds.tsx +++ b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/itemKinds.tsx @@ -17,6 +17,7 @@ import {parseGatewayEntry} from "../toolUtils" import { describeMcp, describeSkill, + describeSubagent, describeTool, isEmbedRefSkill, isFunctionTool, @@ -50,14 +51,18 @@ export interface ItemKindDef { FormView: ItemFormView /** Drawer header title for the current draft. */ drawerTitle: (draft: Record) => string - /** Wider drawer for kinds that need it (skills are two-pane). */ - drawerWidth?: number - /** Full-bleed body so the Form can lay out its own master/detail (the tool parameter editor). */ - formFlush?: boolean + /** Wider drawer. Per ITEM: one kind holds both a two-pane editor and a plain panel. */ + drawerWidth?: (item: Record) => number | undefined + /** Full-bleed body, for a Form that lays out its own master/detail. Per ITEM, as above. */ + formFlush?: (item: Record) => boolean /** Default Form/JSON view when opening an existing item. */ editView: (item: unknown) => ConfigItemView /** Items with no structured form open JSON-only (no Form/JSON toggle). */ jsonOnly: (item: Record) => boolean + /** The item's form already states its identity, so the drawer drops its header chrome. */ + statesOwnIdentity?: (item: Record) => boolean + /** Hide the Form/JSON toggle for an item whose raw shape is an internal detail. */ + formOnly?: (item: Record) => boolean /** Read-only items (e.g. static `__ag__*` skills) — viewable but not editable. */ isReadOnly: (item: unknown) => boolean /** Seed for a fresh "create" draft. */ @@ -75,10 +80,12 @@ export const ITEM_KINDS: Record = { emptyLabel: "No tools yet", describe: describeTool, FormView: ToolFormView, - // Two-panel parameter master/detail — needs width + a full-bleed body. - drawerWidth: 800, - formFlush: true, + // Only the two-pane parameter editor wants width and its own padding. + drawerWidth: (draft) => (isReferenceTool(draft) ? undefined : 800), + formFlush: (draft) => !isReferenceTool(draft), drawerTitle: (draft) => { + // A subagent's header is the agent's NAME. describeTool would call it a workflow. + if (isReferenceTool(draft)) return describeSubagent(draft).name const name = describeTool(draft).name return name && name !== "Tool" ? name : "New tool" }, @@ -92,6 +99,9 @@ export const ITEM_KINDS: Record = { return isFunctionTool(item) || isReferenceTool(item) || entry ? "form" : "json" }, jsonOnly: (draft) => ITEM_KINDS.tool.editView(draft) === "json", + // A subagent's detail states its own identity and hides the raw entry. + statesOwnIdentity: (draft) => isReferenceTool(draft), + formOnly: (draft) => isReferenceTool(draft), isReadOnly: () => false, // Unused for tools: creation seeds from the picker (buildInlineFunctionTool), not this. createSeed: () => ({}), @@ -156,7 +166,7 @@ export const ITEM_KINDS: Record = { describe: describeSkill, FormView: SkillFormView, // Wider than the default 600 — the skill drawer is two-pane (Files + editor). - drawerWidth: 760, + drawerWidth: () => 760, drawerTitle: (draft) => isEmbedRefSkill(draft) ? "Skill reference" diff --git a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/subagentReference.ts b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/subagentReference.ts new file mode 100644 index 0000000000..d4b4678b1c --- /dev/null +++ b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/subagentReference.ts @@ -0,0 +1,14 @@ +/** The one shape a saved subagent may take. A subagent always runs the target agent's latest + * revision, so a legacy pin (`version`, `environment`) or a forbidden `variant_id` is dropped + * on every write rather than left where no surface can show or clear it. */ +export function normalizeSubagentReference(tool: Record): Record { + const { + variant_id: _variantId, + version: _version, + environment: _environment, + ref_by: _refBy, + type: _type, + ...rest + } = tool + return {type: "reference", ref_by: "variant", ...rest} +} diff --git a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/useAgentTools.ts b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/useAgentTools.ts index cb9999832d..d660357eab 100644 --- a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/useAgentTools.ts +++ b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/useAgentTools.ts @@ -9,7 +9,6 @@ import type {WorkflowReferenceBridge, WorkflowReferencePayload} from "@agenta/ui import {migrateIntegration} from "../gatewayMigration" import {DEFAULT_INTEGRATION_PERMISSIONS, mergeToolPermission} from "../integrationPolicy" -import type {ToolSelectionMeta} from "../ToolSelectorPopover" import { buildIntegrationRows, findGatewayConnectionIndex, @@ -21,11 +20,11 @@ import { type GatewayConnectionTarget, type GatewayPermission, type IntegrationRow, - type ToolObj, } from "../toolUtils" -import {isBuiltinPayloadMatch, toolName, toolReferenceSlug} from "./itemDescriptors" +import {toolReferenceSlug} from "./itemDescriptors" import type {ItemKind} from "./itemKinds" +import {normalizeSubagentReference} from "./subagentReference" export function useAgentTools({ config, @@ -53,38 +52,7 @@ export function useAgentTools({ [config, onChange], ) - const handleAddTool = useCallback( - (tool: ToolObj, meta?: ToolSelectionMeta) => { - // `needsConfig` is a transient routing flag — never persist it in the tool metadata. - const {needsConfig, ...toolMeta} = meta ?? ({} as ToolSelectionMeta) - const hasMeta = Object.keys(toolMeta).length > 0 - const next = - hasMeta && tool && typeof tool === "object" && !Array.isArray(tool) - ? { - ...(tool as Record), - agenta_metadata: { - ...(((tool as Record).agenta_metadata as - | Record - | undefined) ?? {}), - ...toolMeta, - }, - } - : tool - // Open the config editor (append only on Save) for a custom tool, or a gateway action - // whose input schema couldn't be resolved — so a half-filled/schema-less tool never - // lands silently. Complete gateway tools add straight away (gateway is multi-select). - if (toolMeta.source === "custom" || needsConfig) { - openCreate("tool", next as Record, "form") - return - } - setTools([...tools, next]) - }, - [tools, setTools, openCreate], - ) - - // Append a `type:"reference"` tool for a workflow chosen in the reference drawer (#4860), - // auto-deriving its model-facing input schema from the workflow's latest revision. The axis - // (variant/environment), pinned version, and environment come from the drawer's payload. + // Append a subagent, deriving its model-facing input schema from the target's latest revision. const handleAddWorkflowReference = useCallback( async (payload: WorkflowReferencePayload) => { const wf = workflowReference?.workflows.find((w) => w.slug === payload.slug) @@ -100,52 +68,23 @@ export function useAgentTools({ const latest = configRef.current const latestTools = Array.isArray(latest.tools) ? (latest.tools as unknown[]) : [] if (latestTools.some((t) => toolReferenceSlug(t) === payload.slug)) return - const referenceTool: Record = { - type: "reference", - ref_by: payload.refBy, + const referenceTool = normalizeSubagentReference({ slug: payload.slug, - ...(payload.refBy === "variant" && payload.variant - ? {variant_id: payload.variant} - : {}), - ...(payload.refBy === "variant" && payload.version - ? {version: payload.version} - : {}), - ...(payload.refBy === "environment" && payload.environment - ? {environment: payload.environment} - : {}), name: wf?.name || payload.slug, description: payload.description ?? wf?.description ?? wf?.name ?? "", input_schema: inputSchema ?? {type: "object", properties: {}}, - } + }) onChange({...latest, tools: [...latestTools, referenceTool]}) }, [workflowReference, onChange, configRef], ) - const handleRemoveToolByName = useCallback( - (name: string) => setTools(tools.filter((tool) => toolName(tool) !== name)), - [tools, setTools], - ) - - const handleRemoveBuiltinTool = useCallback( - (toolToRemove: ToolObj) => { - let removed = false - const updated = tools.filter((tool) => { - if (removed) return true - if (!isBuiltinPayloadMatch(tool, toolToRemove)) return true - removed = true - return false - }) - if (removed) setTools(updated) - }, + // Removal by SLUG: a reference's display name is editable and can match another tool. + const handleRemoveReferenceBySlug = useCallback( + (slug: string) => setTools(tools.filter((tool) => toolReferenceSlug(tool) !== slug)), [tools, setTools], ) - const selectedToolNames = useMemo( - () => new Set(tools.map(toolName).filter((n): n is string => Boolean(n))), - [tools], - ) - // ── Integrations: one `gateway_connection` entry per provider and integration ──────────── const integrationRows = useMemo(() => buildIntegrationRows(tools), [tools]) @@ -207,22 +146,10 @@ export function useAgentTools({ [tools, setTools], ) - // Workflows not yet referenced as a tool — the pool the selector drawer offers. - const referenceableWorkflows = useMemo(() => { - const referenced = new Set( - tools.map((t) => toolReferenceSlug(t)).filter((s): s is string => Boolean(s)), - ) - return (workflowReference?.workflows ?? []).filter((w) => !referenced.has(w.slug)) - }, [tools, workflowReference]) - return { tools, - handleAddTool, handleAddWorkflowReference, - handleRemoveToolByName, - handleRemoveBuiltinTool, - selectedToolNames, - referenceableWorkflows, + handleRemoveReferenceBySlug, integrationRows, setIntegrationConnection, setIntegrationPermissions, diff --git a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/useModelHarness.tsx b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/useModelHarness.tsx index bd52ba3bc1..3712ce51e9 100644 --- a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/useModelHarness.tsx +++ b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/useModelHarness.tsx @@ -149,6 +149,7 @@ export function useModelHarness({ // is harness-filtered: selecting a model sets BOTH the model id and its provider, fed by the // `/inspect` capability map below. const harnessValue = effectiveHarnessValue(harness) + // "pi_agenta" is a removed experiment; old stored revisions may still carry it. const isPiHarness = harnessValue === "pi_core" || harnessValue === "pi_agenta" const llm = config.llm const modelId = useMemo(() => modelIdFromConfig(llm), [llm]) diff --git a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/harnessMeta.ts b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/harnessMeta.ts index 604855595c..aaf14248ea 100644 --- a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/harnessMeta.ts +++ b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/harnessMeta.ts @@ -16,16 +16,16 @@ export interface HarnessMeta { /** * Avatar identity (brand colour + monogram) per harness id. Labels come from the schema `oneOf` * title when present; these defaults only supply the avatar and a label fallback. Keyed by the real - * enum values `pi_core` / `pi_agenta` / `claude`. + * enum values `pi_core` / `claude` / `codex`. */ export const HARNESS_META: Record = { pi_core: {label: "Pi", short: "Pi", color: "#6b5bd6"}, - pi_agenta: {label: "Pi (Agenta)", short: "Ag", color: "#1c2c3d"}, claude: {label: "Claude Code", short: "CC", color: "#d97757"}, codex: {label: "Codex", short: "Cx", color: "#10a37f"}, } -/** Harnesses never offered in a picker. */ +/** Harnesses never offered in a picker. `pi_agenta` (a removed experiment) stays listed so a + * web build in front of an older API that still advertises it never shows it. */ export const HIDDEN_HARNESSES = new Set(["pi_agenta"]) /** Resolve display identity, deriving a sensible fallback for unknown harness ids. */ diff --git a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/hooks/useFamilyMap.ts b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/hooks/useFamilyMap.ts new file mode 100644 index 0000000000..f304a2b217 --- /dev/null +++ b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/hooks/useFamilyMap.ts @@ -0,0 +1,14 @@ +import {useMemo} from "react" + +import {atom, useAtomValue, type Atom} from "jotai" + +/** Read one atom-family entry per key in one subscription. Keys join into a string so the + * memo has one stable dependency; an array would rebuild the atom every render. */ +export function useFamilyMap(keysKey: string, family: (key: string) => Atom): Map { + const derived = useMemo(() => { + const keys = keysKey ? keysKey.split("\n") : [] + return atom((get) => keys.map((key) => [key, get(family(key))] as const)) + }, [keysKey, family]) + const pairs = useAtomValue(derived) + return useMemo(() => new Map(pairs), [pairs]) +} diff --git a/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/hooks/useIntegrationLogos.ts b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/hooks/useIntegrationLogos.ts new file mode 100644 index 0000000000..7e9e875bd8 --- /dev/null +++ b/web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/hooks/useIntegrationLogos.ts @@ -0,0 +1,27 @@ +import {useMemo} from "react" + +import {toolIntegrationDetailQueryFamily} from "@agenta/entities/gatewayTool" + +import {useFamilyMap} from "./useFamilyMap" + +export interface IntegrationMark { + key: string + name: string + logo: string | null +} + +const logoFamily = (key: string) => toolIntegrationDetailQueryFamily(key) + +/** Resolve a set of integration keys to their brand name and logo, in one subscription. */ +export function useIntegrationLogos(keys: string[]): Map { + const keysKey = useMemo(() => [...new Set(keys)].filter(Boolean).sort().join("\n"), [keys]) + const byKey = useFamilyMap(keysKey, logoFamily) + return useMemo(() => { + const marks = new Map() + for (const [key, res] of byKey) { + const catalog = res?.data?.integration + marks.set(key, {key, name: catalog?.name ?? key, logo: catalog?.logo ?? null}) + } + return marks + }, [byKey]) +} diff --git a/web/packages/agenta-entity-ui/src/DrillInView/bridges/useWorkflowReferenceBridge.ts b/web/packages/agenta-entity-ui/src/DrillInView/bridges/useWorkflowReferenceBridge.ts index 6ba2b331a5..1d73262d07 100644 --- a/web/packages/agenta-entity-ui/src/DrillInView/bridges/useWorkflowReferenceBridge.ts +++ b/web/packages/agenta-entity-ui/src/DrillInView/bridges/useWorkflowReferenceBridge.ts @@ -12,14 +12,10 @@ */ import {useMemo} from "react" -import {appEnvironmentsQueryAtomFamily} from "@agenta/entities/environment" import type {RunnablePort} from "@agenta/entities/shared" import { discardLocalServerDataAtom, - evaluatorTemplatesMapAtom, nonArchivedWorkflowsAtom, - parseWorkflowKeyFromUri, - queryWorkflowRevisionsByWorkflow, resolveInputSchema as resolveWorkflowInputSchema, resolveOutputSchema as resolveWorkflowOutputSchema, resolveParameters, @@ -34,64 +30,21 @@ import { import {projectIdAtom} from "@agenta/shared/state" import {KNOWN_ENVELOPE_SLOTS} from "@agenta/shared/utils" import type { + SubagentDetail, WorkflowConfigPart, WorkflowConfigPayload, - WorkflowEnvironmentUI, WorkflowReferenceBridge, + WorkflowReferenceCatalogEntry, WorkflowReferenceType, - WorkflowReferenceUI, - WorkflowRevisionUI, } from "@agenta/ui/drill-in" import {atom, getDefaultStore, useAtomValue, useSetAtom, useStore} from "jotai" import {atomFamily} from "jotai/utils" import {atomWithQuery} from "jotai-tanstack-query" -// A workflow's revisions, fetched on demand when one is selected in the reference drawer (the -// variant-axis version picker). Keyed by workflow id; the project is singular in scope. -const workflowRevisionsQueryAtomFamily = atomFamily((workflowId: string) => - atomWithQuery((get) => { - const projectId = get(projectIdAtom) - return { - queryKey: ["agentWorkflowRevisions", workflowId, projectId], - queryFn: () => queryWorkflowRevisionsByWorkflow(workflowId, projectId as string), - enabled: Boolean(workflowId) && Boolean(projectId), - staleTime: 60_000, - } - }), -) - -function useWorkflowRevisions(workflow: WorkflowReferenceUI | null): { - revisions: WorkflowRevisionUI[] - isLoading: boolean -} { - const res = useAtomValue(workflowRevisionsQueryAtomFamily(workflow?.id ?? "")) - const revisions = useMemo(() => { - const list = (res.data?.workflow_revisions ?? []) as Record[] - return list - .map((r) => ({ - version: r.version != null ? String(r.version) : "", - label: typeof r.message === "string" ? (r.message as string) : undefined, - })) - .filter((r) => Boolean(r.version) && Number(r.version) > 0) - .sort((a, b) => Number(b.version) - Number(a.version)) - }, [res.data]) - return {revisions, isLoading: Boolean(res.isLoading)} -} - -function useWorkflowEnvironments(workflow: WorkflowReferenceUI | null): { - environments: WorkflowEnvironmentUI[] - isLoading: boolean -} { - const res = useAtomValue(appEnvironmentsQueryAtomFamily(workflow?.id ?? "")) - const environments = useMemo( - () => - (res.data ?? []) - .filter((env) => Boolean(env.slug)) - .map((env) => ({slug: env.slug, name: env.name || env.slug})), - [res.data], - ) - return {environments, isLoading: Boolean(res.isLoading)} -} +import {describeSkill} from "../SchemaControls/agentTemplate/itemDescriptors" +import {connectionFromConfig, modelIdFromConfig} from "../SchemaControls/connectionUtils" +import {integrationPermissionSummary} from "../SchemaControls/integrationPolicy" +import {buildIntegrationRows} from "../SchemaControls/toolUtils" // Map the molecule's canonical workflow type down to the four the reference picker badges. function toReferenceType(t: WorkflowType | null | undefined): WorkflowReferenceType | undefined { @@ -118,12 +71,68 @@ function classifyRevision(revision: Workflow): WorkflowReferenceType | undefined interface ReferenceTypeInfo { type: WorkflowReferenceType | undefined - /** For evaluators: the evaluator template key (from the revision URI), for a finer badge label. */ - evaluatorKey: string | null + /** The workflow this revision belongs to. A row's icon is keyed by workflow id. */ + workflowId: string | null + /** What the picker's rows show, read from the revision this query already fetched. */ + model: string | null + provider: string | null + /** Integration keys this agent has connected, e.g. ["github", "slack"]. */ + integrations: string[] + /** This slug's revision fetch failed. Without it the agent silently leaves the picker. */ + failed?: boolean +} + +/** The workflow a revision belongs to. */ +function workflowIdOf(revision: Workflow): string | null { + const id = (revision as unknown as Record).workflow_id + return typeof id === "string" && id ? id : null +} + +/** The model, provider and connected apps of one workflow's latest revision. Every field is + * optional: a missing one means the workflow has none, never that the request failed. */ +function summarizeRevision(revision: Workflow): { + model: string | null + provider: string | null + integrations: string[] +} { + const cfg = agentConfigOf(revision) + if (!cfg) return {model: null, provider: null, integrations: []} + // Agents nest the model under `llm`; prompt-shaped configs use `llm_config`. + const llm = isPlainRecord(cfg.llm) + ? cfg.llm + : isPlainRecord(cfg.llm_config) + ? cfg.llm_config + : null + const model = llm ? modelIdFromConfig(llm.model ?? llm) : null + const provider = llm ? connectionFromConfig(llm).provider : null + const tools = Array.isArray(cfg.tools) ? (cfg.tools as unknown[]) : [] + // The same parser the config panel uses, so picker and panel cannot disagree. + const integrations = buildIntegrationRows(tools).map((row) => row.integration) + return {model, provider, integrations} +} + +/** The agent template inside a revision. NOT on `data` directly: `resolveParameters` unwraps the + * envelope and the config then sits flat or one level down under its own key. */ +function agentConfigOf(revision: Workflow): Record | null { + const params = resolveParameters(revision.data as Parameters[0]) + if (!isPlainRecord(params)) return null + if (isPromptLike(params)) return params + for (const value of Object.values(params)) { + if (isPlainRecord(value) && isPromptLike(value)) return value + } + return null +} + +const EMPTY_REFERENCE_INFO: ReferenceTypeInfo = { + type: undefined, + workflowId: null, + model: null, + provider: null, + integrations: [], } -// Resolve type + evaluator-key for a set of workflow slugs. Keyed by the sorted slug set so the batch -// is cached and only refetches when the set changes. +// Type, binding and display summary for a set of slugs. One revision fetch per workflow serves +// all of it, keyed by the sorted set; nothing here may add a per-row request. const referenceTypesQueryAtomFamily = atomFamily((slugsKey: string) => atomWithQuery((get) => { const projectId = get(projectIdAtom) @@ -140,18 +149,19 @@ const referenceTypesQueryAtomFamily = atomFamily((slugsKey: string) => projectId: projectId as string, workflowRef: {slug}, }) - if (!revision) return [slug, {type: undefined, evaluatorKey: null}] + if (!revision) return [slug, EMPTY_REFERENCE_INFO] return [ slug, { type: classifyRevision(revision), - evaluatorKey: revision.flags?.is_evaluator - ? parseWorkflowKeyFromUri(revision.data?.uri) - : null, + workflowId: workflowIdOf(revision), + ...summarizeRevision(revision), }, ] } catch { - return [slug, {type: undefined, evaluatorKey: null}] + // Recorded, never silently empty: an unmarked failure drops the agent + // from the picker and caches that absence for five minutes. + return [slug, {...EMPTY_REFERENCE_INFO, failed: true}] } }), ) @@ -161,16 +171,6 @@ const referenceTypesQueryAtomFamily = atomFamily((slugsKey: string) => }), ) -// Humanize an evaluator key as a fallback when the template catalog lacks a display name. -// e.g. "auto_exact_match" → "Exact Match". -function humanizeEvaluatorKey(key: string): string { - return key - .replace(/^(auto|human)_/, "") - .replace(/_/g, " ") - .replace(/\b\w/g, (c) => c.toUpperCase()) - .trim() -} - // Lazy activation for the workflow-reference bridge. Referencing a workflow as an agent tool is // the only consumer of the project-wide workflow list + evaluator catalog inside the always-mounted // playground drill-in provider, and it's needed only once the user opens the reference picker or an @@ -181,49 +181,109 @@ const activateWorkflowReferenceAtom = atom(null, (get, set) => { if (!get(workflowReferenceActivatedAtom)) set(workflowReferenceActivatedAtom, true) }) const EMPTY_WORKFLOW_REFS: Workflow[] = [] -const EMPTY_EVALUATOR_NAMES = new Map() const workflowReferenceWorkflowsAtom = atom((get) => get(workflowReferenceActivatedAtom) ? get(nonArchivedWorkflowsAtom) : EMPTY_WORKFLOW_REFS, ) const workflowReferenceLoadingAtom = atom((get) => get(workflowReferenceActivatedAtom) ? get(workflowsListQueryStateAtom).isPending : false, ) -const workflowReferenceEvaluatorNamesAtom = atom((get) => - get(workflowReferenceActivatedAtom) ? get(evaluatorTemplatesMapAtom) : EMPTY_EVALUATOR_NAMES, + +/** One subagent's detail, keyed by slug. Kept out of the picker's batch: 200 agents would carry + * megabytes of instruction text the picker never shows. */ +const subagentDetailQueryAtomFamily = atomFamily((slug: string) => + atomWithQuery((get) => { + const projectId = get(projectIdAtom) + return { + queryKey: ["agentSubagentDetail", projectId, slug], + enabled: Boolean(projectId) && Boolean(slug), + staleTime: 60_000, + queryFn: async () => { + const revision = await retrieveWorkflowRevision({ + projectId: projectId as string, + workflowRef: {slug}, + }) + return revision ? subagentDetailOf(revision) : null + }, + } + }), ) -function useWorkflowReferenceTypes(workflows: WorkflowReferenceUI[]): { - typeBySlug: Record - labelBySlug?: Record +function useSubagentDetail(slug: string): {detail: SubagentDetail | null; loading: boolean} { + const res = useAtomValue(subagentDetailQueryAtomFamily(slug)) + return {detail: (res.data as SubagentDetail | null) ?? null, loading: Boolean(res.isLoading)} +} + +/** Words in a body of prose, for the instruction file's "Markdown, N words" line. */ +function countWords(text: string): number { + const words = text.trim().match(/\S+/g) + return words ? words.length : 0 +} + +/** One subagent's configuration, read off its latest revision. */ +function subagentDetailOf(revision: Workflow): SubagentDetail { + const cfg = agentConfigOf(revision) + const summary = summarizeRevision(revision) + const tools = cfg && Array.isArray(cfg.tools) ? (cfg.tools as unknown[]) : [] + const integrations = buildIntegrationRows(tools).map((row) => ({ + key: row.integration, + // The permission the agent granted this app, in the same words its own row uses. + permission: row.entry + ? integrationPermissionSummary(row.entry.permissions).label + : undefined, + })) + const skills = (cfg && Array.isArray(cfg.skills) ? (cfg.skills as unknown[]) : []) + .map((skill) => describeSkill(skill).name) + .filter(Boolean) + const agentsMd = + cfg && isPlainRecord(cfg.instructions) && typeof cfg.instructions.agents_md === "string" + ? cfg.instructions.agents_md + : null + return { + workflowId: workflowIdOf(revision) ?? undefined, + description: typeof revision.description === "string" ? revision.description : undefined, + model: summary.model ?? undefined, + provider: summary.provider ?? undefined, + integrations, + skills, + instructions: agentsMd + ? {fileName: "AGENTS.md", text: agentsMd, wordCount: countWords(agentsMd)} + : undefined, + } +} + +/** Everything the Subagents picker needs for a batch of workflows, off one cached revision fetch. */ +function useWorkflowReferenceCatalog(slugs: string[]): { + bySlug: Record + failedSlugs: string[] loading: boolean + retry: () => void } { - const slugsKey = useMemo( - () => - workflows - .map((w) => w.slug) - .filter(Boolean) - .sort() - .join("\n"), - [workflows], - ) + // Sorted and joined so the same set of slugs, in any order, hits one cached batch. + const slugsKey = useMemo(() => [...slugs].filter(Boolean).sort().join("\n"), [slugs]) const res = useAtomValue(referenceTypesQueryAtomFamily(slugsKey)) - // Evaluator template catalog (key → display name), for the evaluator sub-type badge. - // Gated behind the bridge activation so it doesn't fire the catalog on a plain playground load. - const evaluatorNames = useAtomValue(workflowReferenceEvaluatorNamesAtom) + const refetch = res.refetch return useMemo(() => { const data = (res.data ?? {}) as Record - const typeBySlug: Record = {} - const labelBySlug: Record = {} + const bySlug: Record = {} + const failedSlugs: string[] = [] for (const [slug, info] of Object.entries(data)) { - typeBySlug[slug] = info.type - if (info.evaluatorKey) { - labelBySlug[slug] = - evaluatorNames.get(info.evaluatorKey) ?? humanizeEvaluatorKey(info.evaluatorKey) + if (info.failed) failedSlugs.push(slug) + bySlug[slug] = { + type: info.type, + workflowId: info.workflowId ?? undefined, + model: info.model ?? undefined, + provider: info.provider ?? undefined, + integrations: info.integrations, } } - return {typeBySlug, labelBySlug, loading: Boolean(res.isLoading)} - }, [res.data, res.isLoading, evaluatorNames]) + return { + bySlug, + failedSlugs, + loading: Boolean(res.isLoading), + retry: () => void refetch(), + } + }, [res.data, res.isLoading, refetch]) } function isPlainRecord(value: unknown): value is Record { @@ -523,7 +583,7 @@ export function useWorkflowReferenceBridge(): WorkflowReferenceBridge { enabled: true, activate, // All project workflows are referenceable (apps + evaluators + …), not just apps. Type - // (incl. `evaluator`) is resolved per-slug via useWorkflowTypes. + // (incl. `evaluator`) is resolved per-slug via useWorkflowReferenceCatalog. workflows: workflows .filter((w) => typeof w.slug === "string") .map((w) => ({ @@ -531,7 +591,7 @@ export function useWorkflowReferenceBridge(): WorkflowReferenceBridge { slug: w.slug as string, name: w.name ?? undefined, description: w.description ?? undefined, - // type is resolved asynchronously via useWorkflowTypes (needs the revision URI). + // type is resolved asynchronously via useWorkflowReferenceCatalog (needs the revision URI). })), workflowsLoading, resolveInputSchema: async (workflow) => { @@ -588,9 +648,8 @@ export function useWorkflowReferenceBridge(): WorkflowReferenceBridge { const parts = buildConfigParts(revision.data) return parts.length ? {parts} : null }, - useWorkflowRevisions, - useWorkflowEnvironments, - useWorkflowTypes: useWorkflowReferenceTypes, + useWorkflowReferenceCatalog, + useSubagentDetail, }), [activate, workflows, workflowsLoading, projectId, store], ) diff --git a/web/packages/agenta-entity-ui/src/DrillInView/index.ts b/web/packages/agenta-entity-ui/src/DrillInView/index.ts index b58653a7cc..c2717ebe01 100644 --- a/web/packages/agenta-entity-ui/src/DrillInView/index.ts +++ b/web/packages/agenta-entity-ui/src/DrillInView/index.ts @@ -118,8 +118,6 @@ export type { WorkflowReferenceBridge, WorkflowReferenceUI, WorkflowReferenceType, - WorkflowRevisionUI, - WorkflowEnvironmentUI, WorkflowReferencePayload, WorkflowConfigPart, WorkflowConfigPayload, @@ -432,6 +430,17 @@ export type {SkillTemplateControlProps} from "./SchemaControls/SkillTemplateCont export {ToolFormView} from "./SchemaControls/ToolFormView" export type {ToolFormViewProps} from "./SchemaControls/ToolFormView" export {ReferenceToolFormView} from "./SchemaControls/ReferenceToolFormView" +export { + SubagentList, + ToolManagementList, + selectSubagentTools, +} from "./SchemaControls/agentTemplate/ToolManagementList" +export {AddSubagentDrawer} from "./SchemaControls/agentTemplate/AddSubagentDrawer" +export type { + SubagentOption, + SubagentIntegration, +} from "./SchemaControls/agentTemplate/AddSubagentDrawer" +export type {SubagentListProps} from "./SchemaControls/agentTemplate/ToolManagementList" export type {ReferenceToolFormViewProps} from "./SchemaControls/ReferenceToolFormView" export {McpServerFormView} from "./SchemaControls/McpServerFormView" export type {McpServerFormViewProps} from "./SchemaControls/McpServerFormView" diff --git a/web/packages/agenta-entity-ui/src/drive/DriveHeader.tsx b/web/packages/agenta-entity-ui/src/drive/DriveHeader.tsx index f0c6cf70a0..b4766607bc 100644 --- a/web/packages/agenta-entity-ui/src/drive/DriveHeader.tsx +++ b/web/packages/agenta-entity-ui/src/drive/DriveHeader.tsx @@ -2,8 +2,10 @@ import {humanSize} from "@agenta/entities/drive" import {type DriveId} from "@agenta/entities/drive" import {fileOrigin} from "@agenta/entities/drive" import {type Mount} from "@agenta/entities/session" +import {shortcutAria} from "@agenta/shared/utils" import {CopyButton} from "@agenta/ui/components/presentational" import {Tag, EnhancedButton as Button} from "@agenta/ui/components/presentational" +import {ShortcutKeys} from "@agenta/ui/shortcuts" import { DropdownMenu, DropdownMenuContent, @@ -115,9 +117,22 @@ export const DriveHeader = ({ : "border-colorBorderSecondary py-2" }`} > - + + Collapse files + + ) : ( + "Close" + ) + } + > + + + + ) +} diff --git a/web/packages/agenta-ui/src/shortcuts/index.ts b/web/packages/agenta-ui/src/shortcuts/index.ts new file mode 100644 index 0000000000..b51e020db2 --- /dev/null +++ b/web/packages/agenta-ui/src/shortcuts/index.ts @@ -0,0 +1,13 @@ +export { + useSessionShortcuts, + isAltChord, + SESSION_SHORTCUT_MAX, + type UseSessionShortcutsParams, +} from "./useSessionShortcuts" +export {ShortcutKeys, useIsMacPlatform, type ShortcutKeysProps} from "./ShortcutKeys" +export { + KeyboardShortcutsSheet, + useShortcutsSheetHotkey, + type KeyboardShortcutsSheetProps, +} from "./KeyboardShortcutsSheet" +export {ShortcutsHelpButton, type ShortcutsHelpButtonProps} from "./ShortcutsHelpButton" diff --git a/web/oss/src/components/AgentChatSlice/hooks/useSessionShortcuts.ts b/web/packages/agenta-ui/src/shortcuts/useSessionShortcuts.ts similarity index 73% rename from web/oss/src/components/AgentChatSlice/hooks/useSessionShortcuts.ts rename to web/packages/agenta-ui/src/shortcuts/useSessionShortcuts.ts index a1e1f7e59c..b03e97b5cf 100644 --- a/web/oss/src/components/AgentChatSlice/hooks/useSessionShortcuts.ts +++ b/web/packages/agenta-ui/src/shortcuts/useSessionShortcuts.ts @@ -1,5 +1,7 @@ import {useEffect} from "react" +import {isOverlayOpen} from "@agenta/shared/utils" + /** How many open sessions the digit row can reach. */ export const SESSION_SHORTCUT_MAX = 9 @@ -15,6 +17,7 @@ export interface UseSessionShortcutsParams { onCloseSession: (id: string) => void onSearch: () => void onToggleConfigPanel: () => void + onToggleFilesPane: () => void } /** A bare Alt chord: no AltGr (Ctrl+Alt), no Cmd or Shift, not a repeat or an IME keystroke. @@ -22,15 +25,6 @@ export interface UseSessionShortcutsParams { export const isAltChord = (e: KeyboardEvent): boolean => e.altKey && !e.ctrlKey && !e.metaKey && !e.shiftKey && !e.repeat && !e.isComposing -/** True while an antd confirm/modal or a Radix dialog owns the screen. No global open-dialog state - * exists to ask, and these dialogs come from `modal.confirm`, so the DOM is the only witness. */ -export const isOverlayOpen = (): boolean => - Boolean( - document.querySelector( - '.ant-modal-wrap:not([style*="display: none"]), [role="dialog"][data-state="open"]', - ), - ) - /** Physical keys that step through the strip, one session at a time. Z and X sit directly above * Alt/Option, so the whole set stays under one resting hand — the reason they're positions * (`event.code`), not letters, on a non-QWERTY layout. */ @@ -50,16 +44,26 @@ const steppedSession = ( } /** - * Session shortcuts for the agent playground: `Alt+1…9` jumps to the Nth open session, `Alt+Z` and - * `Alt+X` step to the previous/next one (wrapping), `Alt+C` opens a new session, `Alt+W` closes the - * active one, `Alt+R` renames it, `Alt+A` archives it, `Alt+F` searches, `Alt+B` toggles the config - * panel. Stop and approve live with the conversation that owns the run, not here. + * Session shortcuts for a chat surface that keeps sessions in a strip. Every action arrives as a + * callback, so the hook knows nothing about its host: the desktop playground drives it today, and + * `/m` can drive the same one when it grows a keyboard surface. A phone never sends an Alt chord, + * so mounting it there is inert rather than harmful. + * + * The bindings: `Alt+1…9` jumps to the Nth open session, `Alt+Z` and + * `Alt+X` step to the previous/next one (wrapping), `Alt++` opens a new session, `Alt+W` closes the + * active one, `Alt+R` renames it, `Alt+A` archives it, `Alt+K` searches, `Alt+C` toggles the config + * panel, `Alt+O` toggles the files pane. Stop and approve live with the conversation that owns the + * run, not here. * * Alt alone, because ⌘/Ctrl+digit is browser tab switching on every OS, and one binding for all * platforms (the label differs, the keys don't). Matched on `event.code`: macOS Option+1 reports * `event.key` as `¡`. Excluding `ctrlKey` keeps European AltGr (which reports as Ctrl+Alt) typing * normally. These fire from any focus context, the composer included — that's the point of a * modifier combo here, and no plain-key binding is introduced that could swallow typed text. + * + * The letters avoid every browser menu mnemonic: Chrome and Edge open their menu on `Alt+F`/`Alt+E`, + * Firefox opens File/Edit/View/History/Bookmarks/Tools/Help on `Alt+F/E/V/S/B/T/H`, and both focus + * the address bar on `Alt+D`. Search moved off `F` and the config panel off `B` for that reason. */ export function useSessionShortcuts({ sessions, @@ -72,6 +76,7 @@ export function useSessionShortcuts({ onCloseSession, onSearch, onToggleConfigPanel, + onToggleFilesPane, }: UseSessionShortcutsParams) { useEffect(() => { if (!enabled) return @@ -104,21 +109,31 @@ export function useSessionShortcuts({ return } - if (e.code === "KeyC") { + // The `+` key, matching the button in the tab strip. Not a letter: macOS makes + // Option+N the tilde dead key, so binding N would eat `ñ` in the composer. + if (e.code === "Equal") { claim() onNewSession() return } - if (e.code === "KeyF") { + if (e.code === "KeyK") { claim() onSearch() return } - if (e.code === "KeyB") { + if (e.code === "KeyC") { claim() onToggleConfigPanel() return } + // The files pane is per-session: with no active session the panel renders none, so + // toggling would flip a state nothing shows. + if (e.code === "KeyO") { + if (!activeId) return + claim() + onToggleFilesPane() + return + } // The rest act on the active session. Closing the last one would leave the panel to // re-seed an empty tab, so it needs a sibling to fall back to. @@ -150,5 +165,6 @@ export function useSessionShortcuts({ onCloseSession, onSearch, onToggleConfigPanel, + onToggleFilesPane, ]) } diff --git a/web/oss/src/components/AgentChatSlice/hooks/useSessionShortcuts.test.ts b/web/packages/agenta-ui/tests/unit/useSessionShortcuts.render.test.ts similarity index 69% rename from web/oss/src/components/AgentChatSlice/hooks/useSessionShortcuts.test.ts rename to web/packages/agenta-ui/tests/unit/useSessionShortcuts.render.test.ts index d0813e6b80..8555d76d24 100644 --- a/web/oss/src/components/AgentChatSlice/hooks/useSessionShortcuts.test.ts +++ b/web/packages/agenta-ui/tests/unit/useSessionShortcuts.render.test.ts @@ -1,24 +1,21 @@ +// @vitest-environment jsdom /** - * Unit tests for the playground session shortcuts (Alt+1…9 / Alt+Z / Alt+X / Alt+R / Alt+A) and the - * inline-rename consumer they drive. + * Unit tests for the session shortcuts. The hook takes every action as a callback and knows + * nothing about its host, so these drive it directly with spies. * * The bindings are matched on `event.code`, not `event.key`: macOS reports Option+1 as `¡` and * Option+R as `®`. The negative cases matter as much as the positive ones — Ctrl+Alt is AltGr on * European layouts (typing `³`, `€`), so a match there would eat characters in the composer. */ -import {act, createElement, useRef} from "react" +import {act, createElement} from "react" -import {chatPanelMaximizedAtom} from "@agenta/chat/state" -import {getDefaultStore} from "jotai" import {createRoot, type Root} from "react-dom/client" import {afterEach, describe, expect, it, vi} from "vitest" -import type {SessionTabLabelHandle} from "../components/SessionTabLabel" -import {AgentChatScopeProvider} from "../state/scope" -import {renameSessionRequestAtom} from "../state/uiRequests" - -import {useInlineRenameRequest} from "./useInlineRenameRequest" -import {useSessionShortcuts, type UseSessionShortcutsParams} from "./useSessionShortcuts" +import { + useSessionShortcuts, + type UseSessionShortcutsParams, +} from "../../src/shortcuts/useSessionShortcuts" const sessions = [{id: "s1"}, {id: "s2"}, {id: "s3"}] @@ -37,6 +34,7 @@ const setup = (overrides: Partial = {}) => { onCloseSession: vi.fn(), onSearch: vi.fn(), onToggleConfigPanel: vi.fn(), + onToggleFilesPane: vi.fn(), } const Probe = () => { useSessionShortcuts({sessions, activeId: "s2", ...handlers, ...overrides}) @@ -121,16 +119,39 @@ describe("useSessionShortcuts", () => { expect(onJump).not.toHaveBeenCalled() }) - it("opens, closes, searches and toggles the config panel", () => { - const {onNewSession, onCloseSession, onSearch, onToggleConfigPanel} = setup() - press("KeyC") + it("opens, closes, searches and toggles both side panels", () => { + const {onNewSession, onCloseSession, onSearch, onToggleConfigPanel, onToggleFilesPane} = + setup() + press("Equal") press("KeyW") - press("KeyF") - press("KeyB") + press("KeyK") + press("KeyC") + press("KeyO") expect(onNewSession).toHaveBeenCalled() expect(onCloseSession).toHaveBeenCalledWith("s2") expect(onSearch).toHaveBeenCalled() expect(onToggleConfigPanel).toHaveBeenCalled() + expect(onToggleFilesPane).toHaveBeenCalled() + }) + + // The letters that browsers claim: Chrome/Edge open their menu on Alt+F, Firefox opens a menu + // on Alt+F/E/V/S/B/T/H, and both focus the address bar on Alt+D. None may be bound here. + it("binds no letter a browser menu already claims", () => { + const handlers = setup() + for (const code of ["KeyF", "KeyE", "KeyV", "KeyS", "KeyB", "KeyT", "KeyH", "KeyD"]) { + press(code) + } + for (const handler of Object.values(handlers)) { + expect(handler).not.toHaveBeenCalled() + } + }) + + // The pane is per-session and the panel renders none without one, so the key must not flip a + // state nothing shows. + it("refuses to toggle the files pane with no active session", () => { + const {onToggleFilesPane} = setup({sessions: [], activeId: undefined}) + press("KeyO") + expect(onToggleFilesPane).not.toHaveBeenCalled() }) it("refuses to close the last remaining session", () => { @@ -215,64 +236,3 @@ describe("useSessionShortcuts", () => { expect(unmatched.defaultPrevented).toBe(false) }) }) - -/** - * The rename request reaches two mounted rows for one session (the strip chip and the rail row), - * so only the one on screen may open its editor. The hidden one still claims the nonce, or - * maximizing later would replay a stale request. - */ -describe("useInlineRenameRequest", () => { - const SCOPE = "app-1" - const store = getDefaultStore() - let renameRoot: Root | null = null - const startEditing = vi.fn() - - const mountRow = (surface: "strip" | "rail") => { - const Row = () => { - const ref = useRef({startEditing}) - useInlineRenameRequest("s1", ref, surface) - return null - } - const host = document.createElement("div") - document.body.append(host) - renameRoot = createRoot(host) - act(() => { - renameRoot?.render( - createElement(AgentChatScopeProvider, {scopeKey: SCOPE}, createElement(Row)), - ) - }) - } - const requestRename = (nonce: number, scope = SCOPE) => - act(() => { - store.set(renameSessionRequestAtom, {scope, sessionId: "s1", nonce}) - }) - - afterEach(() => { - act(() => renameRoot?.unmount()) - renameRoot = null - store.set(renameSessionRequestAtom, null) - store.set(chatPanelMaximizedAtom, false) - startEditing.mockClear() - }) - - it("opens the editor on the surface that is on screen", () => { - mountRow("strip") - requestRename(1) - expect(startEditing).toHaveBeenCalledTimes(1) - }) - - it("stays shut on the off-screen surface, and does not replay when it becomes visible", () => { - mountRow("rail") // off screen while the chat is not maximized - requestRename(1) - act(() => { - store.set(chatPanelMaximizedAtom, true) - }) - expect(startEditing).not.toHaveBeenCalled() - }) - - it("ignores another scope's request for the same session id", () => { - mountRow("strip") - requestRename(1, "drawer:app-1") - expect(startEditing).not.toHaveBeenCalled() - }) -}) diff --git a/web/pnpm-lock.yaml b/web/pnpm-lock.yaml index 5d776dd083..ffb9c8a6e1 100644 --- a/web/pnpm-lock.yaml +++ b/web/pnpm-lock.yaml @@ -2409,6 +2409,9 @@ importers: storybook: dependencies: + '@agenta/chat': + specifier: workspace:* + version: link:../packages/agenta-chat '@agenta/entities': specifier: workspace:* version: link:../packages/agenta-entities diff --git a/web/storybook/package.json b/web/storybook/package.json index 48fee66c87..3d0f045345 100644 --- a/web/storybook/package.json +++ b/web/storybook/package.json @@ -2,7 +2,7 @@ "name": "@agenta/storybook", "private": true, "version": "0.0.0", - "description": "Storybook workbench for the antd \u2192 @agenta/ui migration. Renders real @agenta/* components behind the real app providers so antd and @agenta/ui versions can be compared side by side.", + "description": "Storybook workbench for the antd → @agenta/ui migration. Renders real @agenta/* components behind the real app providers so antd and @agenta/ui versions can be compared side by side.", "scripts": { "storybook": "storybook dev -p 6006", "build-storybook": "storybook build -o storybook-static", @@ -11,6 +11,7 @@ "a11y": "node parity/a11y.mjs" }, "dependencies": { + "@agenta/chat": "workspace:*", "@agenta/entities": "workspace:*", "@agenta/entity-ui": "workspace:*", "@agenta/oss": "workspace:*", diff --git a/web/storybook/stories/domain/KeyboardShortcuts.stories.tsx b/web/storybook/stories/domain/KeyboardShortcuts.stories.tsx new file mode 100644 index 0000000000..150c4dcf12 --- /dev/null +++ b/web/storybook/stories/domain/KeyboardShortcuts.stories.tsx @@ -0,0 +1,264 @@ +/** + * The proposal from the shortcut map, made real enough to press. + * + * Every story here renders shipping components, not sketches: the approval card is the real + * `ApprovalCard`, the sheet and its button are the real `ShortcutsHelpButton`, and the two panel + * tooltips read their keys from the same registry the handlers are written against. + */ +import {ApprovalCard} from "@agenta/chat/components" +import type {PendingApproval} from "@agenta/chat/model" +import {shortcutAria} from "@agenta/shared/utils" +import {KeyboardShortcutsSheet, ShortcutKeys, ShortcutsHelpButton} from "@agenta/ui/shortcuts" +import {Button, SimpleTooltip} from "@agenta/ui/ui" +import {CaretDoubleLeft, CaretDoubleRight, GearSix, Robot} from "@phosphor-icons/react" +import type {Meta, StoryObj} from "@storybook/nextjs" + +const meta = { + title: "@agenta/ui/Domain/KeyboardShortcuts", + component: ShortcutsHelpButton, + subcomponents: {KeyboardShortcutsSheet, ShortcutKeys}, + parameters: { + layout: "padded", + docs: { + description: { + component: + "Forty-three keyboard shortcuts already ship in the agent playground, and six of them tell you they exist. This adds two more, the files pane and the sheet itself, so the registry lists forty-five.\n\n`/m` renders the same `ApprovalCard` with `touch` set, so its keycaps are suppressed; see the **On mobile** story. The shortcut layer itself is desktop-only today, but every piece of it now lives in a package `/m` already depends on: the registry and the overlay guard in `@agenta/shared/utils`, and `useSessionShortcuts`, `ShortcutKeys`, `KeyboardShortcutsSheet` and `ShortcutsHelpButton` in `@agenta/ui/shortcuts`. A mobile host wires the same callbacks; nothing needs reimplementing. These stories show where the rest become visible.\n\nTwo layers: keys on the control that already does the job, and one sheet on `?` for the shortcuts no control can carry. The letters avoid every browser menu key, so the same bindings work on Windows, Linux and macOS.\n\n**Used in:** 1 place — the playground top bar (`PlaygroundHeader`), rightmost after the settings gear.", + }, + }, + }, +} satisfies Meta +export default meta +type Story = StoryObj + +const APPROVAL: PendingApproval[] = [ + { + approvalId: "apr-1", + toolName: "GITHUB_CREATE_ISSUE", + input: { + owner: "Agenta-AI", + repo: "agenta", + title: "Playground shortcut hints", + body: "Surface the keyboard bindings on the controls that already do the job.", + }, + }, +] + +const noop = () => undefined + +const Frame = ({ + title, + note, + children, +}: { + title: string + note: string + children: React.ReactNode +}) => ( +
+

{title}

+

{note}

+
{children}
+
+) + +/** + * The approval card is the surface where a keyboard answer is genuinely faster than a mouse, and + * where a mis-press costs the most, so the keys stay on screen instead of hiding in a tooltip. + */ +export const ApprovalCardKeys: Story = { + render: () => ( + + + + ), +} + +/** What `/m` renders at phone width: the same components, with every keycap gone. */ +export const OnMobile: Story = { + render: () => ( +
+ +
+ +
+ + + +
+
+ +
+
+ +
+
+ +
+ ), +} + +/** + * The sheet and its button ship together: a hotkey with no button teaches nobody. + */ +export const ShortcutsSheet: Story = { + render: () => ( +
+ +
+ + + + + + Refund agent + + + + + + +
+ + +

+ Try it now: +

+ +
+ ), +} + +/** The keys ride on affordances the reader already hovers, on both halves of every toggle. */ +export const PanelTooltips: Story = { + render: () => ( +
+ +
+ + Show configuration + + } + > + + + + Show files + + } + > + + +
+ + + +
+ {[ + {label: "Rename", id: "session.rename"}, + {label: "Pin", id: undefined}, + {label: "Archive", id: "session.archive"}, + {label: "Delete", id: undefined}, + {label: "Close", id: "session.close"}, + {label: "Close other tabs", id: undefined}, + ].map((row) => ( + // Mirrors the real ContextMenuItem: a full-width flex row, so a + // misalignment here is the same misalignment the app would show. +
+ {row.id ? ( + + {row.label} + + + ) : ( + row.label + )} +
+ ))} +
+ + + + + New session + + } + > + + + +
+ ), +} diff --git a/web/storybook/stories/entity-ui/AddSubagentDrawer.stories.tsx b/web/storybook/stories/entity-ui/AddSubagentDrawer.stories.tsx new file mode 100644 index 0000000000..e1f96f70ec --- /dev/null +++ b/web/storybook/stories/entity-ui/AddSubagentDrawer.stories.tsx @@ -0,0 +1,218 @@ +import {AddSubagentDrawer, type SubagentOption} from "@agenta/entity-ui/drill-in" +import type {Meta, StoryObj} from "@storybook/nextjs" + +// The "pick the agents this agent can call" surface, sharing CatalogListRow, +// ExpandableDescription and LogoMarks with the integration drawer. +const meta = { + title: "@agenta/entity-ui/DrillIn/AddSubagentDrawer", + component: AddSubagentDrawer, + parameters: { + layout: "fullscreen", + docs: { + description: { + component: + "Add one or many agents as subagents. Each row adds itself and removes " + + "itself with the same button, Add all acts on what the search is " + + "showing, and the footer only closes.", + }, + }, + }, +} satisfies Meta + +export default meta +type Story = StoryObj + +const noop = () => undefined + +// Copied from the generated agent-icon catalog, which is a 1300-line module. +const GLYPH = { + headphones: + '', + "pen-nib": + '', + "magnifying-glass": + '', + shield: '', + "chart-line": + '', + bug: '', + translate: + '', +} as const +/** Palette entries from AGENT_ICON_COLORS, so the tinted chips match the picker's own swatches. */ +const icon = (name: keyof typeof GLYPH, color: string) => ({name, color, path: GLYPH[name]}) + +// Inline marks, never a remote CDN: a story that reaches the network fails in CI. +const LOGO = { + github: "data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%20viewBox%3D%220%200%2024%2024%22%3E%3Crect%20width%3D%2224%22%20height%3D%2224%22%20rx%3D%226%22%20fill%3D%22%2324292F%22%2F%3E%3Ctext%20x%3D%2212%22%20y%3D%2217%22%20font-family%3D%22system-ui%2Csans-serif%22%20font-size%3D%2214%22%20font-weight%3D%22600%22%20fill%3D%22%23fff%22%20text-anchor%3D%22middle%22%3EG%3C%2Ftext%3E%3C%2Fsvg%3E", + slack: "data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%20viewBox%3D%220%200%2024%2024%22%3E%3Crect%20width%3D%2224%22%20height%3D%2224%22%20rx%3D%226%22%20fill%3D%22%234A154B%22%2F%3E%3Ctext%20x%3D%2212%22%20y%3D%2217%22%20font-family%3D%22system-ui%2Csans-serif%22%20font-size%3D%2214%22%20font-weight%3D%22600%22%20fill%3D%22%23fff%22%20text-anchor%3D%22middle%22%3ES%3C%2Ftext%3E%3C%2Fsvg%3E", + linear: "data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%20viewBox%3D%220%200%2024%2024%22%3E%3Crect%20width%3D%2224%22%20height%3D%2224%22%20rx%3D%226%22%20fill%3D%22%235E6AD2%22%2F%3E%3Ctext%20x%3D%2212%22%20y%3D%2217%22%20font-family%3D%22system-ui%2Csans-serif%22%20font-size%3D%2214%22%20font-weight%3D%22600%22%20fill%3D%22%23fff%22%20text-anchor%3D%22middle%22%3EL%3C%2Ftext%3E%3C%2Fsvg%3E", + notion: "data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%20viewBox%3D%220%200%2024%2024%22%3E%3Crect%20width%3D%2224%22%20height%3D%2224%22%20rx%3D%226%22%20fill%3D%22%230F0F0F%22%2F%3E%3Ctext%20x%3D%2212%22%20y%3D%2217%22%20font-family%3D%22system-ui%2Csans-serif%22%20font-size%3D%2214%22%20font-weight%3D%22600%22%20fill%3D%22%23fff%22%20text-anchor%3D%22middle%22%3EN%3C%2Ftext%3E%3C%2Fsvg%3E", +} + +/** Past two lines at the drawer's width, which is what makes the Show more toggle appear. */ +const LONG_DESCRIPTION = + "Reads the full ticket thread including every attachment and prior escalation, decides the " + + "severity against the current on-call policy, names the owning team, drafts the internal " + + "summary, and posts it to the right channel so the on-call engineer has context before they " + + "open the ticket." + +const OPTIONS: SubagentOption[] = [ + { + id: "wf-1", + name: "Support triage", + description: + "Reads an incoming support ticket, decides its severity, and names the team that owns it.", + icon: {...icon("headphones", "#0E7490"), icon: "headphones"}, + model: "claude-sonnet-4-5", + provider: "Anthropic", + integrations: [ + {key: "linear", name: "Linear", logo: LOGO.linear}, + {key: "slack", name: "Slack", logo: LOGO.slack}, + ], + }, + { + id: "wf-2", + name: "Reply drafter", + description: "Writes a first-draft reply in the team's voice, ready for a human to edit.", + icon: {...icon("pen-nib", "#7C3AED"), icon: "pen-nib"}, + model: "claude-opus-4-1", + provider: "Anthropic", + integrations: [{key: "notion", name: "Notion", logo: null}], + }, + { + id: "wf-3", + name: "Release researcher", + description: + "Searches the changelog and the open issues, then summarizes what shipped since a given version.", + icon: {...icon("magnifying-glass", "#1668DC"), icon: "magnifying-glass"}, + model: "gpt-5", + provider: "OpenAI", + integrations: [{key: "github", name: "GitHub", logo: LOGO.github}], + }, + { + id: "wf-4", + name: "Bug reproducer", + description: + "Turns a bug report into a minimal reproduction and reports whether it still fails.", + icon: {...icon("bug", "#D61010"), icon: "bug"}, + model: "claude-sonnet-4-5", + provider: "Anthropic", + integrations: [], + }, + { + // No icon of its own: the card falls back to the robot glyph on a neutral chip. + id: "wf-5", + name: "Metrics reporter", + description: "Pulls last week's numbers and writes the short version for the standup.", + model: "gpt-5-mini", + provider: "OpenAI", + integrations: [{key: "github", name: "GitHub", logo: LOGO.github}], + }, + { + id: "wf-6", + name: "Translator", + description: "Rewrites a message in another language without losing the original tone.", + icon: {...icon("translate", "#389E0D"), icon: "translate"}, + model: "claude-sonnet-4-5", + provider: "Anthropic", + integrations: [], + added: true, + }, +] + +// The drawer positions itself; `data-vrt-subject` is the harness's readiness marker. +const Frame = (children: React.ReactNode) => ( +
+ {children} +
+) + +/** Every row state the list can produce, in one screen. */ +export const Default: Story = { + args: {open: true, onClose: noop, options: OPTIONS, onAdd: noop, onRemove: noop}, + render: (args) => Frame(), +} + +/** A short list, which is what a young project actually looks like. */ +export const FewAgents: Story = { + args: {open: true, onClose: noop, options: OPTIONS.slice(0, 2), onAdd: noop, onRemove: noop}, + render: (args) => Frame(), +} + +/** Nothing to add yet. The copy points at the one thing that fixes it. */ +export const NoAgents: Story = { + args: {open: true, onClose: noop, options: [], onAdd: noop, onRemove: noop}, + render: (args) => Frame(), +} + +/** First paint, before the project's agents resolve. */ +export const Loading: Story = { + args: {open: true, onClose: noop, options: [], loading: true, onAdd: noop, onRemove: noop}, + render: (args) => Frame(), +} + +/** The clamp both ways, plus the row's worst case: long name, long model, four apps. */ +export const LongContent: Story = { + args: { + open: true, + onClose: noop, + onAdd: noop, + onRemove: noop, + options: [ + { + id: "wf-long-a", + name: "Customer escalation triage and routing assistant for the support organization", + description: LONG_DESCRIPTION, + icon: {...icon("headphones", "#CA8A04"), icon: "headphones"}, + model: "claude-sonnet-4-5-20250929-preview", + provider: "Anthropic", + integrations: [ + {key: "github", name: "GitHub", logo: LOGO.github}, + {key: "slack", name: "Slack", logo: LOGO.slack}, + {key: "linear", name: "Linear", logo: LOGO.linear}, + {key: "notion", name: "Notion", logo: null}, + ], + }, + { + id: "wf-long-b", + name: "Support triage", + description: LONG_DESCRIPTION, + icon: {...icon("headphones", "#0E7490"), icon: "headphones"}, + model: "claude-sonnet-4-5", + provider: "Anthropic", + integrations: [ + {key: "linear", name: "Linear", logo: LOGO.linear}, + {key: "slack", name: "Slack", logo: LOGO.slack}, + ], + }, + ], + }, + render: (args) => Frame(), +} + +/** Every agent is already added, so every row offers Remove and the header drops Add all. */ +export const AllAlreadyAdded: Story = { + args: { + open: true, + onClose: noop, + onAdd: noop, + onRemove: noop, + options: OPTIONS.map((o) => ({...o, added: true})), + }, + render: (args) => Frame(), +} + +/** Two agents failed to load. The list says so and offers a retry, rather than hiding them. */ +export const SomeFailedToLoad: Story = { + args: { + open: true, + onClose: noop, + onAdd: noop, + onRemove: noop, + options: OPTIONS.slice(0, 3), + failedCount: 2, + onRetry: noop, + }, + render: (args) => Frame(), +} diff --git a/web/storybook/stories/entity-ui/AgentTemplateControl.stories.tsx b/web/storybook/stories/entity-ui/AgentTemplateControl.stories.tsx index 66165a94b1..f141c1f629 100644 --- a/web/storybook/stories/entity-ui/AgentTemplateControl.stories.tsx +++ b/web/storybook/stories/entity-ui/AgentTemplateControl.stories.tsx @@ -14,13 +14,15 @@ import { GraduationCap, Plugs, Plus, + PuzzlePiece, + Robot, SlidersHorizontal, - Wrench, } from "@phosphor-icons/react" import type {Meta, StoryObj} from "@storybook/nextjs" import {Button as AntButton, Tooltip as AntTooltip, Typography as AntTypography} from "antd" import type {StoryScope} from "../../.storybook/decorators/withAgentaData" +import {integrationQueries, GITHUB_WORK, SLACK_OPS} from "../../fixtures/gatewayIntegration" // AgentTemplateControl — the agent playground's left config panel and the composition root of // `SchemaControls/agentTemplate/*`. Its own antd surface was small (three `Tooltip` + icon @@ -117,8 +119,19 @@ const AGENT_VALUE = { }, harness: {kind: "claude_code"}, tools: [ - {name: "web_search", description: "Search the web"}, - {name: "read_file", description: "Read a file from the workspace"}, + { + type: "gateway_connection", + connection: {provider: "composio", integration: "github", slug: GITHUB_WORK.slug ?? ""}, + policy: {permissions: {default: "allow", tools: {}}}, + }, + { + type: "reference", + name: "triage_ticket", + slug: "support-triage", + ref_by: "version", + version: "3", + description: "Reads a support ticket and returns its severity and owning team.", + }, ], mcps: [{name: "linear", url: "https://mcp.linear.app/sse"}], skills: [{name: "release-notes", description: "Draft release notes from a changelog"}], @@ -150,8 +163,10 @@ const HARNESS_CATALOG = { }, } -const agentQueries = (_scope: StoryScope): [readonly unknown[], unknown][] => [ +// The integration queries come along so the Integrations rows resolve their app name and logo. +const agentQueries = (scope: StoryScope): [readonly unknown[], unknown][] => [ [["workflows", "catalog", "harnesses"], HARNESS_CATALOG], + ...integrationQueries(scope, {connections: [GITHUB_WORK, SLACK_OPS]}), ] // --------------------------------------------------------------------------- @@ -174,7 +189,7 @@ const renderPanel = (schema: unknown, value: unknown, disabled?: boolean) => (
) -/** Everything configured: model + harness, instructions, two tools, an MCP server, a skill. */ +/** Everything configured: model + harness, instructions, an integration, a subagent, an MCP server, a skill. */ export const Configured: Story = { args: { schema: AGENT_SCHEMA as SchemaProperty, @@ -274,12 +289,21 @@ const SECTIONS: AgentTemplateSectionDescriptor[] = [ }, { key: "tools", - icon: , - title: "Tools", - summary: "2 tools", - extra: , + icon: , + title: "Integrations", + summary: "1 integration", + extra: , + defaultOpen: true, + content: Github, + }, + { + key: "subagents", + icon: , + title: "Subagents", + summary: "1 subagent", + extra: , defaultOpen: true, - content: web_search · read_file, + content: triage_ticket, }, { key: "mcp", @@ -321,8 +345,8 @@ export const SectionList: Story = {
diff --git a/web/storybook/stories/entity-ui/AgentToolSelectorPopover.stories.tsx b/web/storybook/stories/entity-ui/AgentToolSelectorPopover.stories.tsx deleted file mode 100644 index d58fd02319..0000000000 --- a/web/storybook/stories/entity-ui/AgentToolSelectorPopover.stories.tsx +++ /dev/null @@ -1,195 +0,0 @@ -import {useEffect, useRef, type ReactNode} from "react" - -import type {GatewayToolsBridge} from "@agenta/ui/drill-in" -import {Plus, Wrench} from "@phosphor-icons/react" -import type {Meta, StoryObj} from "@storybook/nextjs" -import {Button as AntButton} from "antd" - -// Imported from source: the DrillInView barrel does not re-export the agent-scoped picker. -import {AgentToolSelectorPopover} from "../../../packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/AgentToolSelectorPopover" - -// AgentToolSelectorPopover — the agent-playground "+ Tool" picker (Approach B): a thin grouped -// `AddItemMenu` that hands off to dedicated drawers instead of the legacy cascade. Its only antd -// was the default trigger button; the panel itself is the shared (already-migrated) AddItemMenu. -// -// antd swap: `Button variant="outlined" color="default" size="small" icon={}` → -// `@agenta/ui` `Button variant="outline" size="sm"` with the icon as a child. (antd v6 resolves -// `variant="outlined" color="default"` to the plain outlined button — the same trigger the -// shared `ToolSelectorPopover` renders.) -// -// OPEN STATE: `AddItemMenu` exposes no `defaultOpen`/`container`, so the panel cannot be -// portaled inline for a pixel pair — `OpenState` below opens it NATURALLY (Radix-managed) so -// the a11y audit covers the real open state; the panel's own pixel pair lives on -// `agenta-entity-ui-drawers-additemmenu--antd-vs-agenta`. -const meta = { - title: "@agenta/entity-ui/DrillIn/AgentToolSelectorPopover", - component: AgentToolSelectorPopover, - parameters: { - layout: "padded", - docs: { - description: { - component: - "Agent-scoped tool picker: 'Add existing' (reference a workflow · third-party integration) and 'Create new' (tool definition · create with AI, disabled). No built-in provider tools here — that is the legacy prompt playground's picker.", - }, - }, - }, -} satisfies Meta - -export default meta -type Story = StoryObj - -const noop = () => undefined - -const gatewayBridge: GatewayToolsBridge = { - enabled: true, - connections: [], - connectionsLoading: false, - onOpenCatalog: noop, - renderIntegrationInfo: () => ({name: "GitHub"}), - useActions: () => ({ - actions: [], - total: 0, - isLoading: false, - isFetchingNextPage: false, - hasNextPage: false, - requestMore: noop, - setSearch: noop, - prefetchThreshold: 5, - }), - buildToolSlug: (provider, integration, action, connectionSlug) => - `${provider}__${integration}__${action}__${connectionSlug}`, - fetchActionDetail: async () => ({action: {description: "", schemas: {inputs: {}}}}), -} - -// --------------------------------------------------------------------------- -// Parity grid — the closed trigger (the only antd this component carried) -// --------------------------------------------------------------------------- - -const Row = ({ - label, - a, - s, - expected, -}: { - label: string - a: ReactNode - s: ReactNode - expected?: string -}) => ( -
-
{label}
-
- antd -
- {a} -
-
-
- agenta -
- {s} -
-
-
-) - -/** Closed state: the default "+ Tool" trigger, its disabled variant, and a custom trigger. */ -export const AntdVsAgenta: Story = { - render: () => ( -
- } - > - Tool - - } - s={} - /> - } - disabled - > - Tool - - } - s={} - /> - } - className="!h-5 !px-1" - /> - } - s={ - - - - } - /> - } - /> -
- ), -} - -// Opens the popover the way a user would (a real click on the Radix trigger), so the audited -// tree is the one Radix manages — no forced `open` prop, no inert wrapper artifacts. -function AutoOpen({children}: {children: ReactNode}) { - const ref = useRef(null) - useEffect(() => { - ref.current?.querySelector("button")?.click() - }, []) - return ( -
- {children} -
- ) -} - -/** Naturally-opened panel: both groups, the drawer chevrons, and the disabled "Create with AI". */ -export const OpenState: Story = { - render: () => ( - - - - ), -} - -/** Without a gateway bridge only "Create new" renders — the group list is data-driven. */ -export const CreateOnly: Story = { - render: () => ( - - - - ), -} diff --git a/web/storybook/stories/entity-ui/HarnessSelectControl.stories.tsx b/web/storybook/stories/entity-ui/HarnessSelectControl.stories.tsx index fbdacf86c7..3331c5f3db 100644 --- a/web/storybook/stories/entity-ui/HarnessSelectControl.stories.tsx +++ b/web/storybook/stories/entity-ui/HarnessSelectControl.stories.tsx @@ -33,19 +33,19 @@ type Story = StoryObj const HARNESS_META: Record = { pi_core: {label: "Pi", short: "Pi", color: "#6b5bd6"}, - pi_agenta: {label: "Pi (Agenta)", short: "Ag", color: "#1c2c3d"}, claude: {label: "Claude Code", short: "CC", color: "#d97757"}, + codex: {label: "Codex", short: "Cx", color: "#10a37f"}, } const SCHEMA = { type: "string", title: "Harness", description: "The runtime that executes the agent.", - enum: ["pi_core", "pi_agenta", "claude"], + enum: ["pi_core", "claude", "codex"], oneOf: [ {const: "pi_core", title: "Pi"}, - {const: "pi_agenta", title: "Pi (Agenta)"}, {const: "claude", title: "Claude Code"}, + {const: "codex", title: "Codex"}, ], } as never @@ -197,12 +197,12 @@ export const AntdVsAgenta: Story = { /> } + a={} s={ undefined} disabled /> diff --git a/web/storybook/stories/entity-ui/ReferenceToolFormView.stories.tsx b/web/storybook/stories/entity-ui/ReferenceToolFormView.stories.tsx index 14f71b962d..8358f4b3dc 100644 --- a/web/storybook/stories/entity-ui/ReferenceToolFormView.stories.tsx +++ b/web/storybook/stories/entity-ui/ReferenceToolFormView.stories.tsx @@ -1,24 +1,9 @@ -import type {ReactNode} from "react" - -import {RailField} from "@agenta/entity-ui/drawers/shared" -import {ReferenceToolFormView, SchemaTree} from "@agenta/entity-ui/drill-in" -import {ConfigAccordionSection, CopyButton} from "@agenta/ui/components/presentational" -import {GitBranch, Info, TreeStructure} from "@phosphor-icons/react" +import {DrillInUIProvider, ReferenceToolFormView} from "@agenta/entity-ui/drill-in" +import type {SubagentDetail, WorkflowReferenceBridge} from "@agenta/ui/drill-in" import type {Meta, StoryObj} from "@storybook/nextjs" -import {Input as AntInput} from "antd" -// ReferenceToolFormView — the detail view for a `type:"reference"` workflow tool (#4860): -// exposed name, description, the resolved input schema, and the "Reference by" axis. -// Storybook mounts it without a `workflowReference` bridge, so the read-only binding -// summary renders (the editable axis needs the host's bridge). -// -// The antd half replays the pre-migration body verbatim from feat/storybook-data-seam; the -// shared chrome (ConfigAccordionSection / SchemaTree / CopyButton / RailField) is the SAME -// component in both halves, so the diff isolates the migrated leaves. -// -// antd swaps: `Input.TextArea autoSize` → `AutosizeTextarea` (`@agenta/ui`); -// `Spin size="small"` → `Spinner size="small"` (the environment picker's loading slot, -// only reachable with the bridge injected). +// The detail panel for one saved subagent: the calling agent owns only the description. +// A subagent always runs the latest revision, so there is no version control here. const meta = { title: "@agenta/entity-ui/DrillIn/ReferenceToolFormView", component: ReferenceToolFormView, @@ -27,7 +12,9 @@ const meta = { docs: { description: { component: - "Edit counterpart of the WorkflowReferenceSelector: exposed tool name, editable description, read-only input schema, and the reference binding.", + "One saved subagent: an editable description over a read-only summary of the " + + "agent it points at. The instruction file clamps to four lines and expands " + + "into a fixed scrolling well rather than pushing the panel off screen.", }, }, }, @@ -38,165 +25,89 @@ type Story = StoryObj const noop = () => undefined -const INPUT_SCHEMA = { - type: "object", - properties: { - thread: {type: "string", description: "The support thread to summarize"}, - max_bullets: {type: "integer"}, - }, - required: ["thread"], -} - -const PINNED_TOOL = { +const TOOL = { type: "reference", ref_by: "variant", - slug: "summarizer", - version: "3", - description: "Summarize a support thread into three bullets", - input_schema: INPUT_SCHEMA, -} - -const DEPLOYED_TOOL = { - type: "reference", - ref_by: "environment", - slug: "summarizer", - environment: "production", - description: "", - input_schema: null, + slug: "support-triage", + name: "Support triage", + description: "Summarize a support thread into three bullets.", + input_schema: {type: "object", properties: {}}, } -/** Pre-migration body, verbatim (antd baseline). */ -const AntdBody = ({ - slug, - description, - schema, - binding, -}: { - slug: string - description: string - schema: Record | null - binding: string -}) => ( -
-
- } - title="Details" - > - -
- {slug} - -
- +/** Long enough to clamp, which is what makes the Show more link appear. */ +const AGENTS_MD = [ + "You are a QA fixture agent. Run the migration browser matrix, compare legacy and modern", + "rendering paths, and report failures with a reproduction link. Always start from the pinned", + "fixture list. When a combination fails, capture the console output, the rendered screenshot,", + "and the diff against the last known-good run before you report it.", + "", + "Never retry a failing combination more than twice. A third failure is a real defect and it", + "belongs in the report, not in another retry.", +].join("\n") - - - - +const DETAIL: SubagentDetail = { + workflowId: "01a04000-0000-7000-8000-000000000001", + name: "Support triage", + description: "Reads an incoming support ticket and names the team that owns it.", + model: "claude-sonnet-4-5", + provider: "anthropic", + integrations: [ + {key: "github", name: "GitHub", permission: "Allow all"}, + {key: "linear", name: "Linear", permission: "Ask for write and delete"}, + ], + skills: ["Browser matrix run", "Visual diff", "Repro reducer"], + instructions: {fileName: "AGENTS.md", text: AGENTS_MD, wordCount: 2140}, +} - } - title="Schema" - summary={`Inputs · ${schema?.properties ? Object.keys(schema.properties).length : 0}`} - summaryCollapsedOnly - > -
- -
- +/** A bridge that resolves one subagent. Only the members this panel touches are real. */ +const bridgeWith = (detail: SubagentDetail | null) => + ({ + enabled: true, + workflows: [], + workflowsLoading: false, + useSubagentDetail: () => ({detail, loading: false}), + agentHref: (id: string) => `/agents/${id}`, + }) as unknown as WorkflowReferenceBridge - } - title="Reference by" - summary={binding} - summaryCollapsedOnly - > -

{binding}

- -
+const Frame = (bridge: WorkflowReferenceBridge | undefined, children: React.ReactNode) => ( +
+ + {children} +
) -const Row = ({ - label, - a, - s, - expected, -}: { - label: string - a: ReactNode - s: ReactNode - expected?: string -}) => ( -
-
{label}
-
- antd -
- {a} -
-
-
- agenta -
- {s} -
-
-
-) +/** The full panel: identity, the one editable field, and the read-only configuration. */ +export const Default: Story = { + args: {value: TOOL, onChange: noop}, + render: (args) => Frame(bridgeWith(DETAIL), ), +} + +/** An agent with nothing configured. Each row says so rather than rendering an empty gap. */ +export const NothingConfigured: Story = { + args: {value: TOOL, onChange: noop}, + render: (args) => + Frame( + bridgeWith({ + ...DETAIL, + model: undefined, + integrations: [], + skills: [], + instructions: undefined, + }), + , + ), +} -export const AntdVsAgenta: Story = { - args: {value: PINNED_TOOL, onChange: noop}, - render: () => ( -
- - } - s={} - /> - - } - s={} - /> -
- ), +/** No host bridge at all, so nothing resolves. The description still edits: it is local. */ +export const WithoutBridge: Story = { + args: {value: TOOL, onChange: noop}, + render: (args) => Frame(undefined, ), } -/** Read-only (committed revision): the description textarea takes the disabled skin. */ +/** Read-only revision: the description cannot be edited either. */ export const Disabled: Story = { - args: {value: PINNED_TOOL, onChange: noop, disabled: true}, - render: () => ( -
- -
- ), + args: {value: TOOL, onChange: noop, disabled: true}, + render: (args) => + Frame(bridgeWith(DETAIL), ), } diff --git a/web/storybook/stories/entity-ui/SubagentList.stories.tsx b/web/storybook/stories/entity-ui/SubagentList.stories.tsx new file mode 100644 index 0000000000..e4c116102b --- /dev/null +++ b/web/storybook/stories/entity-ui/SubagentList.stories.tsx @@ -0,0 +1,108 @@ +import {SubagentList} from "@agenta/entity-ui/drill-in" +import type {Meta, StoryObj} from "@storybook/nextjs" + +// A subagent is saved as `{type: "reference"}`: the wire format keeps the old name. +const meta = { + title: "@agenta/entity-ui/DrillIn/SubagentList", + component: SubagentList, + parameters: { + layout: "padded", + docs: { + description: { + component: + "The Subagents section body: the published workflows this agent can call, " + + "as a flat row list. Each row's subtitle names the referenced workflow and " + + "how it is pinned, either to a version or to an environment. The list draws " + + "no sub-header and no add button: the accordion section header owns the " + + "title, the count, and the plus.", + }, + }, + }, +} satisfies Meta + +export default meta +type Story = StoryObj + +const noop = () => undefined + +/** Pinned to one variant. `ref_by` is "variant" or "environment", never "version". */ +const variantRef = ( + name: string, + slug: string, + variant: string, + version: string, + description?: string, +) => ({ + type: "reference", + name, + slug, + ref_by: "variant", + variant, + version, + description, +}) + +/** Pinned to an environment, so the agent follows whatever is deployed there. */ +const environmentRef = (name: string, slug: string, environment: string) => ({ + type: "reference", + name, + slug, + ref_by: "environment", + environment, +}) + +const ENTRIES = [ + variantRef( + "triage_ticket", + "support-triage", + "01a04000-0000-7000-8000-000000000001", + "3", + "Reads a support ticket and returns its severity and owning team.", + ), + variantRef( + "summarize_thread", + "thread-summarizer", + "01a04000-0000-7000-8000-000000000002", + "12", + ), + environmentRef("draft_reply", "reply-drafter", "production"), +].map((item, index) => ({item, index})) + +const listArgs = (entries: typeof ENTRIES) => ({ + entries, + openEdit: noop, + removeItem: noop, + closeEditor: noop, + emptyAdd: add a subagent, +}) + +// Showcase, not an antd parity pair. `data-vrt-subject` is the harness's readiness marker. +const Frame = (children: React.ReactNode) => ( +
+ {children} +
+) + +/** Both axes side by side, so a variant row and an environment row can be read against each other. */ +export const RowStates: Story = { + args: listArgs(ENTRIES), + render: (args) => Frame(), +} + +/** One row, which is the common case for an agent that delegates a single job. */ +export const SingleRow: Story = { + args: listArgs(ENTRIES.slice(0, 1)), + render: (args) => Frame(), +} + +/** No subagents yet. The body is one line carrying the add link. */ +export const Empty: Story = { + args: listArgs([]), + render: (args) => Frame(), +} + +/** Read-only revision: no chevron, no Remove, no tab stop, and no empty-state add. */ +export const ReadOnly: Story = { + args: {...listArgs(ENTRIES), disabled: true}, + render: (args) => Frame(), +} diff --git a/web/storybook/stories/entity-ui/ToolFormView.stories.tsx b/web/storybook/stories/entity-ui/ToolFormView.stories.tsx index 16ad07076e..1b3308ef5a 100644 --- a/web/storybook/stories/entity-ui/ToolFormView.stories.tsx +++ b/web/storybook/stories/entity-ui/ToolFormView.stories.tsx @@ -26,7 +26,7 @@ const meta = { docs: { description: { component: - "Structured tool editor. Nothing selected → tool basics (name / description / permission / additionalProperties). A workflow-reference tool routes to ReferenceToolFormView instead.", + "Structured tool editor. Nothing selected → tool basics (name / description / permission / additionalProperties). A subagent routes to ReferenceToolFormView instead, which is a different surface entirely.", }, }, }, @@ -94,7 +94,7 @@ export const GatewayToolWithoutSchema: Story = { render: () => , } -/** `type:"reference"` routes to ReferenceToolFormView (read-only binding without the bridge). */ +/** `type:"reference"` is a SUBAGENT: it routes to ReferenceToolFormView, not to this form. */ export const ReferenceTool: Story = { args: {value: REFERENCE_TOOL, onChange: noop}, render: () => , diff --git a/web/storybook/stories/entity-ui/ToolManagementList.stories.tsx b/web/storybook/stories/entity-ui/ToolManagementList.stories.tsx index e9c62ca17b..19fdf2eca1 100644 --- a/web/storybook/stories/entity-ui/ToolManagementList.stories.tsx +++ b/web/storybook/stories/entity-ui/ToolManagementList.stories.tsx @@ -1,7 +1,6 @@ +import {ToolManagementList} from "@agenta/entity-ui/drill-in" import type {Meta, StoryObj} from "@storybook/nextjs" -// Imported from source: the DrillInView barrel does not re-export the tools list. -import {ToolManagementList} from "../../../packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/ToolManagementList" import {buildIntegrationRows} from "../../../packages/agenta-entity-ui/src/DrillInView/SchemaControls/toolUtils" import type {StoryScope} from "../../.storybook/decorators/withAgentaData" import {integrationQueries, GITHUB_WORK, SLACK_OPS} from "../../fixtures/gatewayIntegration" @@ -16,10 +15,12 @@ const meta = { docs: { description: { component: - "The Tools section's integration rows. Each row summarizes one integration's " + + "The Integrations section body. Each row summarizes one integration's " + "saved policy as a preset label, or as 'Custom · N' when per-tool overrides " + "are saved. An integration still held in the pre-rework per-action format is " + - "tagged 'old format' and shows no policy, because it has none yet.", + "tagged 'old format' and shows no policy, because it has none yet. The list " + + "draws no sub-header and no add button: the accordion section header owns " + + "the title, the count, and the plus.", }, }, }, @@ -59,11 +60,7 @@ const TOOLS = [ const listArgs = (tools: unknown[]) => ({ tools, integrationRows: buildIntegrationRows(tools), - openEdit: noop, - removeItem: noop, - closeEditor: noop, - emptyAdd: null, - onAddIntegration: noop, + emptyAdd: add an integration, onOpenIntegration: noop, onRemoveIntegration: noop, }) @@ -118,3 +115,15 @@ export const ReadOnly: Story = { }, render: (args) => Frame(), } + +/** No integrations yet. The body is one line carrying the add link. */ +export const Empty: Story = { + args: listArgs([]), + render: (args) => Frame(), +} + +/** Read-only and empty: the line disappears rather than offering an add the revision cannot do. */ +export const EmptyReadOnly: Story = { + args: {...listArgs([]), disabled: true}, + render: (args) => Frame(), +} diff --git a/web/storybook/stories/entity-ui/WorkflowReferenceSelector.stories.tsx b/web/storybook/stories/entity-ui/WorkflowReferenceSelector.stories.tsx deleted file mode 100644 index fd7dd78adf..0000000000 --- a/web/storybook/stories/entity-ui/WorkflowReferenceSelector.stories.tsx +++ /dev/null @@ -1,340 +0,0 @@ -import {useState} from "react" - -import type { - WorkflowReferenceBridge, - WorkflowReferenceType, - WorkflowReferenceUI, -} from "@agenta/ui/drill-in" -import { - AutosizeTextarea, - Badge, - Button, - EmptyState, - InputAffix, - Segmented, - Spinner, -} from "@agenta/ui/ui" -import {MagnifyingGlass} from "@phosphor-icons/react" -import type {Meta, StoryObj} from "@storybook/nextjs" -import { - Empty as AntEmpty, - Input as AntInput, - Segmented as AntSegmented, - Spin, - Tag as AntTag, -} from "antd" - -// Not exported from `@agenta/entity-ui/drill-in` (AgentTemplateControl is its only consumer), so -// the story imports the source directly — the relative-import convention used by the @agenta/ui -// primitive stories for unbarrelled components. -import {WorkflowReferenceSelector} from "../../../packages/agenta-entity-ui/src/DrillInView/SchemaControls/WorkflowReferenceSelector" - -// WorkflowReferenceSelector — the master/detail drawer for referencing a workflow as an agent -// tool. Migration: antd `Tag` → `Badge`, antd `Input prefix allowClear` → `InputAffix`, -// antd `Input.TextArea autoSize` → `AutosizeTextarea`, antd `Segmented` → `@agenta/ui` `Segmented`, -// antd `Spin` → `Spinner`, antd `Empty PRESENTED_IMAGE_SIMPLE` → `EmptyState image="simple"`, -// antd `Skeleton` → `@agenta/ui` `Skeleton`. -// -// The drawer portals to `body`, so the state stories are SHOWCASES; `AntdVsAgenta` pairs the -// swapped in-drawer pieces inline against their pre-migration markup -// (`git show feat/storybook-data-seam:web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/WorkflowReferenceSelector.tsx`). -const meta = { - title: "@agenta/entity-ui/DrillIn/WorkflowReferenceSelector", - component: WorkflowReferenceSelector, - parameters: { - layout: "padded", - docs: { - description: { - component: - "Two-panel drawer: a searchable, type-badged workflow rail on the left; the selected workflow's description, exposed tool name, schema and reference axis on the right.", - }, - }, - }, -} satisfies Meta - -export default meta -type Story = StoryObj - -const noop = () => undefined - -const WORKFLOWS: WorkflowReferenceUI[] = [ - { - id: "wf-1", - slug: "summarize_ticket", - name: "Summarize ticket", - description: "Condense a support ticket into three bullets.", - type: "completion", - }, - {id: "wf-2", slug: "chat_agent", name: "Chat agent", type: "chat"}, - {id: "wf-3", slug: "triage_agent", name: "Triage agent", type: "agent"}, - {id: "wf-4", slug: "exact_match", name: "Exact match", type: "evaluator"}, - {id: "wf-5", slug: "custom_scorer", name: "Custom scorer", type: "custom"}, -] - -const TYPE_BY_SLUG: Record = Object.fromEntries( - WORKFLOWS.map((w) => [w.slug, w.type]), -) - -const makeBridge = (over: Partial = {}): WorkflowReferenceBridge => ({ - enabled: true, - workflows: WORKFLOWS, - workflowsLoading: false, - resolveInputSchema: async () => ({ - type: "object", - properties: {ticket: {type: "string", title: "Ticket"}}, - }), - resolveOutputSchema: async () => null, - useWorkflowRevisions: () => ({revisions: [], isLoading: false}), - useWorkflowEnvironments: () => ({environments: [], isLoading: false}), - useWorkflowTypes: () => ({typeBySlug: TYPE_BY_SLUG, loading: false}), - ...over, -}) - -const DrawerDemo = ({ - bridge, - workflows, -}: { - bridge: WorkflowReferenceBridge - workflows?: WorkflowReferenceUI[] -}) => { - const [open, setOpen] = useState(true) - return ( -
- - setOpen(false)} - workflows={workflows ?? WORKFLOWS} - bridge={bridge} - onSelect={noop} - /> -
- ) -} - -/** Resting state — the rail lists every workflow, the detail pane shows the "pick one" hint. */ -export const Default: Story = { - args: {open: true, onClose: noop, workflows: WORKFLOWS, bridge: makeBridge(), onSelect: noop}, - render: () => , -} - -/** Loading — the rail shows the Spinner while the workflow list resolves. */ -export const Loading: Story = { - args: Default.args, - render: () => ( - - ), -} - -/** No referenceable workflows — the rail's EmptyState. */ -export const EmptyList: Story = { - args: Default.args, - render: () => ( - ({typeBySlug: {}, loading: false}), - })} - workflows={[]} - /> - ), -} - -const Row = ({ - label, - a, - s, - expected, -}: { - label: string - a: React.ReactNode - s: React.ReactNode - expected?: string -}) => ( -
-
{label}
-
- antd -
- {a} -
-
-
- agenta -
- {s} -
-
-
-) - -const TYPE_BADGE_CLASS = "max-w-[140px] truncate px-1.5 py-0 text-[10px] leading-[18px]" -const FILTER_OPTIONS = [ - {label: "All", value: "all"}, - {label: "Completion", value: "completion"}, - {label: "Chat", value: "chat"}, - {label: "Agent", value: "agent"}, -] - -export const AntdVsAgenta: Story = { - args: Default.args, - render: () => ( -
- - - agent - - - chat - - - completion - - - custom - - - evaluator - -
- } - s={ -
- - agent - - - chat - - - completion - - - custom - - - evaluator - -
- } - /> - - } - placeholder="Search workflows" - value="triage" - allowClear - /> - } - s={ - - } - placeholder="Search workflows" - aria-label="Search workflows" - value="triage" - onValueChange={noop} - allowClear - /> - } - /> - } - s={ - - } - /> - - -
- } - s={ -
- -
- } - /> - - No workflows to reference - - } - /> - } - s={ - - No workflows to reference - - } - /> - } - /> - - } - s={ - - } - /> -
- ), -} diff --git a/web/storybook/stories/presentational/ShortcutKeys.stories.tsx b/web/storybook/stories/presentational/ShortcutKeys.stories.tsx new file mode 100644 index 0000000000..5d1ea4f949 --- /dev/null +++ b/web/storybook/stories/presentational/ShortcutKeys.stories.tsx @@ -0,0 +1,126 @@ +import {PLAYGROUND_SHORTCUTS, SHORTCUT_GROUP_TITLES, shortcutGroups} from "@agenta/shared/utils" +import {ShortcutKeys} from "@agenta/ui/shortcuts" +import type {Meta, StoryObj} from "@storybook/nextjs" + +// ShortcutKeys — keycaps for one keyboard shortcut, drawn from the shared registry rather than +// from a hand-written string, so a hint can never name a key the handler does not bind. +// +// The platform is read in a mount effect, never during render: the server has no platform and a +// guess mismatches on hydration. Until it lands the caps print the non-Apple faces, which is what +// `isMacPlatform()` already returns server-side. +const meta = { + title: "@agenta/ui/Presentational/Labels/ShortcutKeys", + component: ShortcutKeys, + parameters: { + layout: "padded", + docs: { + description: { + component: + "Keycaps for one keyboard shortcut, printed the way the reader's own keyboard is labelled. The faces come from `PLAYGROUND_SHORTCUTS` in `@agenta/shared/utils`, so a hint can never name a key the handler does not bind. On Apple hardware the caps read `⌘ ⌥ ⌃ ⇧`; everywhere else they read `Ctrl Alt Shift`.\n\n**Used in:** 6 places — the approval card's Approve and Deny buttons, the configuration and files-pane tooltips, the shortcuts help button's tooltip, and every row of the shortcuts sheet.", + }, + }, + }, +} satisfies Meta +export default meta +type Story = StoryObj + +/** The default: one registry id, rendered as the chip tone at the small size. */ +export const Default: Story = {args: {id: "session.new"}} + +/** `chip` sits on a surface. `inverse` sits inside a dark tooltip or a filled primary button. */ +export const Tones: Story = { + render: () => ( +
+
+ chip + + +
+
+ inverse + + +
+
+ size md + + +
+
+ ), +} + +/** + * Decorative against announced. Inside a button the label already names the action, so the caps + * are `aria-hidden` and the key reaches assistive tech through `aria-keyshortcuts` instead. In the + * sheet and in a tooltip the caps ARE the content, so they stay announced. Neither state is + * reachable by clicking, which is why it is a story. + */ +export const Decorative: Story = { + render: () => ( +
+
+ Inside a button + + Deny + + + accessible name stays “Deny” + +
+
+ In the sheet + + Deny + + + + the keys are the content, so announced + +
+
+ ), +} + +/** Every binding the playground ships, straight out of the registry. */ +export const EveryBinding: Story = { + render: () => ( +
+

+ {PLAYGROUND_SHORTCUTS.length} bindings, grouped by the surface that owns them. +

+ {shortcutGroups() + .filter((group) => group.shortcuts.length > 0) + .map((group) => ( +
+

+ {SHORTCUT_GROUP_TITLES[group.id]} +

+ {group.shortcuts.map((shortcut) => ( +
+ + {shortcut.label} + {shortcut.when ? ( + + {" "} + — {shortcut.when} + + ) : null} + + + {shortcut.id} + + +
+ ))} +
+ ))} +
+ ), +}