Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .agents/planner.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,17 @@ Prefer a focused PR stack when independent invariants can be reviewed and
merged separately. Do not split work at a point that requires a temporary
architecture or leaves `main` contradicting its specifications.

## Plan a repetitive migration

Prepare the [parent migration program](../examples/adversarial-implementation/README.md#assess-a-repetitive-migration)
before launching a migration batch. Count one comparable unit across the
parent issue, including completed and planned batches. Name the query or
inventory behind the estimate, the expected batch count and uncertainty.
Refresh the shared record when new evidence changes those facts. If planning
reveals a larger migration, return to launch preparation with the updated
program before handing off implementation. Use the accepted design result and
its representative dry run to settle the rollout in the plan.

## Validate the API before handoff

Implement the smallest API that removes the identified authoring friction.
Expand Down
71 changes: 71 additions & 0 deletions examples/adversarial-implementation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ initial packet within 64000 characters; retain full logs as references.
| `implementation.ready`, `implementation.evidence` | Whether accepted scope and focused checks suffice for direct implementation through existing interfaces. A separate plan document is optional. |
| `sharedApi.needed`, `sharedApi.evidence` | Whether accepted behavior requires shared API work. Necessary API work excludes simple; it alone does not justify a migration split. |
| `migration` | `null` declares no worthwhile separate migration split. A record requires `evidence`, `representativeConsumers` and `remainingConsumers`: repeated production friction, representative validation and substantial identified conversion remaining. Both consumer lists are nonempty, distinct, disjoint repository-relative committed production paths. No file-count threshold applies. |
| `migrationProgram` | Absolute path to the shared parent program for a repetitive migration. Every batch uses it, including batches that use the ordinary route. A non-null `migration` requires it. |

`assessment` is the existing ReviewPolicy impact assessment, independent of
scope. Its evidence and optional risk evidence also select `inputs`. Routing
Expand All @@ -97,6 +98,7 @@ The table is ordered. Missing or unknown facts never become false or null.
| --- | --- |
| Missing/invalid/unknown member, unavailable evidence/source, occupied fresh-work directory, conflicting identities or contradictory migration record | Actionable hold before paid roles; identify what to correct or which existing record to continue |
| Any `decisions.openQuestions` item | Hold before paid roles; show the unanswered question and evidence |
| Parent migration meets the [design cutoff](#assess-a-repetitive-migration) | Obtain or verify the shared Problem Solver design. A held, uncertain or invalid assessment holds the batch before implementation. |
| Valid non-null migration record and `sharedApi.needed=true` | Existing Planner foundation and representative validation, then remaining consumer implementation, selected review and retirement |
| `migration=null`, `sharedApi.needed=false`, `implementation.ready=true` | Direct Implementor, then existing impact-selected independent review |
| `migration=null`, `sharedApi.needed=false`, `implementation.ready=false` | Ordinary Planner plan/evidence, with original Architect discovery when requested, then the same Planner handoff to Implementor and selected review |
Expand All @@ -114,6 +116,10 @@ Any unresolved material question?
| yes -----------------------> HOLD: show question and evidence
| no
v
Parent migration meets design cutoff?
| yes -> obtain or verify shared Problem Solver design
| hold if the result cannot be verified
v
Justified migration record? (requires sharedApi.needed=true)
| yes -----------------------> MIGRATION
| Planner foundation + representatives
Expand Down Expand Up @@ -179,6 +185,71 @@ The roles use the normal agent permission policy. Running this example invokes
paid coding agents and permits the assigned implementation work. Publication,
issue/PR metadata and delivery remain with the Sidekick running Codex.

## Assess a repetitive migration

Use one parent program for all tickets, branches and PRs that make the same
repetitive change. The Planner keeps the program and its supporting query or
inventory outside individual batch checkouts and work directories. Set
`scope.migrationProgram` to the same absolute program path for every batch.
Count comparable sites across the full parent, including completed work.
Count a batch once when it also has a PR. Update the parent record when a
later ticket reveals more sites or changes the rollout.

A program looks like this:

```json
{
"schema": "migration-program.v1",
"parent": "taras/executable.md#1171",
"unit": "one test owner that writes a failure journal",
"estimatedSites": 19,
"plannedBatches": 2,
"inventory": "journal-owners.md",
"uncertainty": {
"additionalSites": 2,
"additionalBatches": 0,
"reason": "Two candidates still need ownership checks."
},
"assumptions": "Each test owner belongs to one of the inventoried harnesses."
}
```

The inventory is a text file beside the program. It records the source
revision, query or counting method, and evidence behind the reported totals.
Use zero for a dimension with no expected additions and `null` for an
unbounded estimate. The workflow requires design at **20 sites or three
batches**. It checks the upper end of both estimates, so the example requires
a design for its 21 possible sites. An unbounded dimension takes the design
path. Below both cutoffs, the workflow checks the two files and makes no
Problem Solver call.

Twenty sites gives a repeatable transformation enough applications to justify
a representative dry run. Three batches catches the fixed planning and
review work repeated on a rollout with fewer sites. These are initial
thresholds. The observed rollout had ten open PRs, 75 commits, about 13,400
non-whitespace changed lines, and 74 test owners pending or partial. A separate
Fable assessment found that 17 rounds each left their own migration and audit
scripts. It recommended consolidating them and backtesting the shared recipe
before delegating routine cases. Token spend was unmeasured.
[Journal attribution migration](https://github.com/taras/executable.md/issues/1171)

The first batch that reaches a cutoff runs the existing Problem Solver design
phase. It compares deterministic edits or a shared helper, batches a cheaper
model can handle, a narrower rollout, and cases that need individual judgment.
A hybrid XMD approach can inventory and preview deterministically, ask a model
to classify ambiguous sites against approved recipes, and apply only the
TypeScript edits whose AST preconditions pass. The assessment recommends a
small dry run and gives distinct, measurable pass and fail outcomes.

The workflow retains one attempt beside the parent record for the exact
program and inventory contents. Further batches using those contents reuse
the verified design and its native session receipt across checkout revisions.
Changed contents require a new assessment when the cutoff applies. Keep the
parent records stable while their assumptions hold. If a reserved attempt is still
running or has no valid result, the batch holds and reports the saved
attempt for reconciliation. Retrying the batch cannot repeat a reserved design turn.
An admitted batch keeps its captured design with the existing Ticket inputs.

## Keep the original conversation

Every production role passes through `Phase.Session` inside its resolved Session,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,7 @@ returns: {type: object}
],
"additionalProperties": false
},
"migrationProgram": {type:"string",pattern:"^/"},
"migration": {
"anyOf": [
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,9 @@ Connect your recommendation to those observations. In `whatWouldChange`, name
the evidence that would change your recommendation. Use `openQuestions` for
missing evidence or a decision the Architect must make. If a missing fact
prevents a choice, recommend the next step and explain what it would resolve.
The Architect owns the implementation contract.
The Architect owns the implementation contract. When the assignment asks for
a dry run, fill in `dryRun` with its procedure and distinct, measurable pass
and fail outcomes.

Return one JSON object. Keep any progress updates in plain prose.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ const designResult = z.strictObject({
recommendation: text,
whatWouldChange: text,
openQuestions: z.array(text).max(12),
dryRun: z.strictObject({ procedure: text, pass: text, fail: text }).optional(),
});
const evidence = z.array(
z.strictObject({
Expand All @@ -86,7 +87,7 @@ const evidence = z.array(
sha256: z.string().regex(/^[a-f0-9]{64}$/),
}),
);
const designInputs = z.array(z.strictObject({ path: text, content: z.string() })).max(12);
export const designInputs = z.array(z.strictObject({ path: text, content: z.string() })).max(12);
const inputs = designInputs.min(1);
const prior = z.strictObject({
schema: z.literal("diagnosis-attempt.v1"),
Expand Down Expand Up @@ -120,7 +121,7 @@ const receipt = z.strictObject({
identity,
result,
});
const designReceipt = receipt.extend({
export const designReceipt = receipt.extend({
assignment: prior.extend({ assignment: design }),
result: designResult,
});
Expand Down
12 changes: 12 additions & 0 deletions examples/adversarial-implementation/components/Entry/Run.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,12 +79,24 @@ test -f {referencePath} -a -r {referencePath}
</Loop>
<If condition={unreadable.length !== 0}><Let as="work" value={{status:"hold",reason:"Prepare readable selected evidence and role-guide texts before launch: " + unreadable.join(", "),history:[]}} /><Break /></If>
<If condition={task.scope.decisions.openQuestions.length !== 0}><Let as="work" value={{status:"hold",reason:"Resolve these material questions before launch: " + task.scope.decisions.openQuestions.join("; ") + ". Evidence: " + task.scope.decisions.evidence.join(", ") + ". Record accepted answers under fresh references and prepare the task again.",history:[]}} /><Break /></If>
<If condition={task.scope.migration !== null && task.scope.migrationProgram === undefined}><Let as="work" value={{status:"hold",reason:"Record the shared parent migration at scope.migrationProgram before launching the API migration.",history:[]}} /><Break /></If>
<Ticket.Documents directory={task.directory + ".inputs"} repository={task.repository} paths={[...task.inputs,...guidePaths]} as="retained" />
<Ticket.Selected repository={task.repository} documents={retained} as="documents" />
<Let as="inputs" value={task.inputs.map(path => documents[path])} />
<Ticket.Capture as="deliveryCheck"><Delivery.Prepare task={task} inputs={inputs} as="delivery" /><Json value={delivery} /></Ticket.Capture>
<If condition={deliveryCheck.status !== "completed"}><Let as="work" value={{status:"hold",reason:deliveryCheck.failure,history:[]}} /><Break /></If>
<Parse schema={{type:"object"}} as="delivery">{deliveryCheck.output}</Parse>
<If condition={task.scope.migrationProgram !== undefined}>
<Ticket.Capture as="designCheck"><Migration.Gate batch={{path:task.scope.migrationProgram,repository:task.repository,repositoryId:task.delivery.repository,directory:task.directory,revision:task.revision}} as="gate" /><Json value={gate} /></Ticket.Capture>
<If condition={designCheck.status !== "completed"}><Let as="work" value={{status:"hold",reason:"Parent migration design gate: " + designCheck.failure,history:[]}} /><Break /></If>
<Parse schema={{type:"object"}} as="gate">{designCheck.output}</Parse>
<If condition={gate.result !== null}>
<Ticket.Documents directory={task.directory + ".inputs"} repository={task.repository} paths={[gate.result]} documents={retained} as="retained" />
<Ticket.Selected repository={task.repository} documents={retained} as="documents" />
<Let as="task" value={{...task,inputs:[...new Set([...task.inputs,gate.result])],request:task.request + "\n\nParent migration " + gate.parent + ". Use the captured Problem Solver recommendation at " + gate.result + ". Start with its dry run if it has not been verified. Resolve material questions through the existing planning gate."}} />
<Let as="inputs" value={task.inputs.map(path => documents[path])} />
</If>
</If>
<Let as="plannerGuide" value={documents[guidePaths[0]]} />
<Let as="implementorGuide" value={documents[guidePaths[1]]} />
<If condition={task.scope.migration !== null}>
Expand Down
40 changes: 40 additions & 0 deletions examples/adversarial-implementation/components/Migration/Gate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
props:
type: object
properties:
batch:
type: object
properties:
path: {type: string, pattern: '^/'}
repository: {type: string, pattern: '^/'}
repositoryId: {type: string}
directory: {type: string, pattern: '^/'}
revision: {type: string, pattern: '^[a-f0-9]{40}$'}
required: [path, repository, repositoryId, directory, revision]
additionalProperties: false
required: [batch]
additionalProperties: false
returns: {type: object}
---

<Let as="batch" value={props.batch} />
<Migration.Program path={batch.path} repository={batch.repository} repositoryId={batch.repositoryId} directory={batch.directory} as="program" />
<If condition={!program.required}><Return value={{result:null}} /><Else>
<Entry.DiagnosticSolvers as="solvers" />
<Diagnosis.Paths repository={batch.repository} attempt={program.attempt} session={program.session} as="resolved" />
<Let as="resultInput" value={{repository:batch.repository,parent:program.parent,directory:resolved.directory,identities:resolved.identities,session:program.session,name:resolved.name,question:program.question,constraints:program.constraints,inputs:program.inputs,candidates:[solvers.primary,...(solvers.fallback ? [solvers.fallback] : [])].map(({key,provider,models,effort}) => ({key,provider,models,effort}))}} />
<Migration.Result input={resultInput} as="saved" />
<If condition={saved === null}>
<Let as="request" value={{role:"problem-solver",session:program.session,attempt:resolved.directory,inputPaths:program.inputs.map(input => input.path),assignment:{kind:"design",work:program.parent,repository:batch.repository,revision:batch.revision,question:program.question,constraints:program.constraints}}} />
<Ticket.Capture as="assessment"><Diagnosis.Run request={request} solvers={solvers} as="review" /><Json value={review} /></Ticket.Capture>
<If condition={assessment.status !== "completed"}><Fail message={"Reconcile the parent migration design at " + resolved.directory + " before starting this batch. " + assessment.failure} /></If>
<Parse schema={{type:"object"}} as="review">{assessment.output}</Parse>
<If condition={review.status !== "design"}><Fail message={"The parent migration design is held at " + review.attempt + ". " + review.reason} /></If>
<Migration.Result input={resultInput} as="saved" />
<If condition={saved === null}><Fail message={"No validated migration design was retained at " + resolved.directory + "."} /></If>
</If>
<Migration.Program path={batch.path} repository={batch.repository} repositoryId={batch.repositoryId} directory={batch.directory} as="current" />
<If condition={program.fingerprint !== current.fingerprint}><Fail message="The parent migration inputs changed during admission. Re-run the batch with the current program." /></If>
<Return value={{result:saved,parent:program.parent}} />
</Else>
</If>
107 changes: 107 additions & 0 deletions examples/adversarial-implementation/components/Migration/Program.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
import { realpath } from "@executablemd/runtime";
import { basename, dirname, isAbsolute, relative, resolve, sep } from "node:path";
import { z } from "zod";
import { describe, read } from "../Ticket/text-storage.ts";

const text = z.string().regex(/\S/).max(4_000);
const count = z.number().int().min(0).max(1_000_000_000);
const absolute = z.string().startsWith("/");
const request = z.strictObject({
path: absolute,
repository: absolute,
repositoryId: text,
directory: absolute,
});
const program = z.strictObject({
schema: z.literal("migration-program.v1"),
parent: z.string().regex(/^[a-zA-Z0-9_.-]+\/[a-zA-Z0-9_.-]+#[1-9][0-9]*$/),
unit: text,
estimatedSites: count,
plannedBatches: count.min(1),
inventory: z.string().regex(/^[a-zA-Z0-9_-][a-zA-Z0-9_.-]*$/),
uncertainty: z.strictObject({
additionalSites: count.nullable(),
additionalBatches: count.nullable(),
reason: text,
}),
assumptions: text,
});

const sites = 20;
const batches = 3;

export const props = z.toJSONSchema(request, { target: "draft-7" });
export const returns = { type: "object" };

function inside(root: string, path: string): boolean {
const part = relative(root, path);
return !isAbsolute(part) && part !== ".." && !part.startsWith(".." + sep);
}

export default function* Program(value: unknown) {
const input = request.parse(value);
const path = yield* realpath(input.path);
const repository = yield* realpath(input.repository);
const directory = yield* realpath(dirname(input.directory));
if (
path === undefined ||
repository === undefined ||
directory === undefined ||
inside(repository, path) ||
inside(resolve(directory, basename(input.directory)), path)
) {
throw new Error(
"Keep the shared parent migration program outside the checkout and batch directory.",
);
}

const content = yield* read(input.repository, path, 64_000);
const recorded = program.parse(JSON.parse(content));
if (recorded.parent.split("#")[0]?.toLowerCase() !== input.repositoryId.toLowerCase()) {
throw new Error("The parent migration issue must belong to the delivery repository.");
}

const inventoryPath = resolve(dirname(path), recorded.inventory);
if (inventoryPath === path || (yield* realpath(inventoryPath)) !== inventoryPath) {
throw new Error(
"Supply the parent migration inventory as a separate regular document beside the program.",
);
}
const inventory = yield* read(input.repository, inventoryPath, 64_000);
if (inventory.trim().length === 0) {
throw new Error("Record the query or inventory supporting the parent migration estimate.");
}

const inputs = [
{ path, content },
{ path: inventoryPath, content: inventory },
];
const fingerprint = describe(
path,
JSON.stringify(inputs.map((entry) => describe(entry.path, entry.content).sha256)),
).sha256;
const { additionalSites, additionalBatches } = recorded.uncertainty;
const required =
additionalSites === null ||
additionalBatches === null ||
recorded.estimatedSites + additionalSites >= sites ||
recorded.plannedBatches + additionalBatches >= batches;
return {
required,
parent: recorded.parent,
fingerprint,
inputs,
attempt: path + ".design-" + fingerprint,
session: "migration-design",
question:
"How can this parent migration deliver reliable results with fewer repeated implementation and review rounds?",
constraints: [
"Use the shared parent program and inventory. Account for completed and planned work and the stated uncertainty.",
"Group comparable units into mechanical edits, known patterns a cheaper model can handle in batches, and cases that need individual judgment.",
"Compare a codemod or shared helper, a cheaper model such as Codex Terra for routine work, a narrower rollout, and where higher judgment is needed.",
"Consider a hybrid XMD program with deterministic inventory and preview, model classification against approved recipes, and TypeScript edits guarded by AST preconditions and focused validation.",
"Recommend a small representative dry run before bulk rollout. Fill in dryRun.procedure, dryRun.pass and dryRun.fail with a measurable check and distinct outcomes.",
"Treat costs without measured token data as estimates. Identify the assumptions that would invalidate the recommendation.",
],
};
}
Loading
Loading