Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 20 additions & 3 deletions TRACKER.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
## Status

- **Current phase:** 6F — Field hardening, feedback round 1 (runs before 6.1–6.5)
- **Next step:** 6F.2 field-pattern eval fixture
- **Done:** 0.1–0.4, 1.1–1.6, 2.1–2.5, 3.1–3.6, 4.1–4.6, 5.1–5.7, 6F.1
- **Next step:** 6F.3 instance→definition resolution hardening
- **Done:** 0.1–0.4, 1.1–1.6, 2.1–2.5, 3.1–3.6, 4.1–4.6, 5.1–5.7, 6F.1–6F.2
- **Gates passed:** Gate 0 (CI + red-path, #5/#6) · Gate 1 (precision 1.000, recall 0.895, zero poison) · Gate 2 (C1 instance attribution 1.000 · B1 4-level handler chains · C6 store writers↔readers · A9 portals — scorecard 137/0/0, precision & recall 1.000) · Gate 3 (B3 action effects · B4 routers · B6 cyclic journeys terminate · B7/B8 form & non-JSX events · G5 flag/role conditions — precision & recall 1.000) · Gate 4 (A4 rarity · A10 fuzzy/OCR · A1 structural · A6 subtree · E3 vision annotations · E2 aliases · G4 corrections — high-conf correct 1.000, ambiguity honesty 1.000, poison rate 0.000) · Gate 5 (F1 context bundle · F2 blast radius · F3 test coverage · F4 response schema · F5 git history · MCP server over stdio — scorecard 265/0/0, all honesty metrics 1.000; **M5 reached** — ticket in → budgeted context bundle out, over MCP)

## What CodeRadar is
Expand Down Expand Up @@ -375,7 +375,7 @@ CalendarPanel): gibberish declines, a real term ranks CalendarPanel top-1 with n
noise, gibberish mixed into a real query doesn't poison it. 8 new core unit tests; eval
271/0/0, gate OK.

### [ ] 6F.2 Field-pattern eval fixture
### [x] 6F.2 Field-pattern eval fixture
**Failure modes:** D2 (the eval blind spot itself)
**Build:** new fixture `eval/fixtures/field-patterns` mirroring the shapes the field app used
and current fixtures miss: multi-hop aliased barrel chains (`index.ts` re-export → tsconfig
Expand All @@ -387,6 +387,23 @@ data-source/route/coverage counts); checks owned by 6F.3–6F.6 land in an expli
are enabled by their steps, so `pnpm eval` stays green throughout.
**Accept:** fixture scans clean; 6F.1 checks green; skip list names the step that must enable
each remaining check.
**Done:** new fixture `eval/fixtures/field-patterns` (11 app files): two-hop barrel + rename
(`components/index.ts` → `ui/index.ts` → definition), `@ui`/`@store/*` tsconfig-path aliases,
`Loadable(lazy(() => import()))` page elements, route arrays declared in `routes.tsx`,
spread-composed, and passed to `createBrowserRouter` as an imported identifier, an RTK Query
store (`createApi` base + two `injectEndpoints` slices with string-form and object-form
`query` plus a `builder.mutation`), and a test rendering through a custom
`renderWithProviders` wrapper. The harness's native xfail is the skip list: 9 target checks
carry `expectedFail` naming the enabling step (6F.3: DataGrid instance count + blast radius ·
6F.4: two endpoint attributions · 6F.5: three routes, one effect, one journey), and
unexpected-pass gating removes stale marks the moment a capability lands. checks.ts change:
xfail-marked attributions stay out of the lineage precision/recall tallies while marked, so a
known-missing capability doesn't depress global metrics. Scanning the fixture confirmed the
field diagnosis — the `@ui` alias import yields NO instance node for UsersPage's `<Grid/>`,
and 0 route/data-source/hook nodes exist. Already working in this shape (kept as passing
checks): relative two-hop barrel rename (InvoicesPage's Grid), covered-by through the custom
wrapper — so 6F.6 must find the actual field-breaking coverage variant and extend the fixture.
eval 280 pass / 0 fail / 9 xfail / 0 unexpected-pass, gate OK, all metrics 1.000.

### [ ] 6F.3 Instance→definition resolution hardening
**Failure modes:** A5, C1, F2, D4
Expand Down
6 changes: 6 additions & 0 deletions eval/fixtures/field-patterns/app/__tests__/UsersPage.test.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
import { renderWithProviders } from "../test-utils";
import UsersPage from "../pages/UsersPage";

it("renders the team directory", () => {
renderWithProviders(<UsersPage />);
});
4 changes: 4 additions & 0 deletions eval/fixtures/field-patterns/app/components/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
// Barrel-of-barrel with a rename — the multi-hop chain the field app used
// everywhere (index.ts → ui/index.ts → definition).
export { DataGrid as Grid } from "./ui";
export { StatusBadge } from "./ui";
24 changes: 24 additions & 0 deletions eval/fixtures/field-patterns/app/components/ui/DataGrid.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import { StatusBadge } from "./StatusBadge";

export function DataGrid({ rows }: { rows: string[] }) {
return (
<table>
<thead>
<tr>
<th>Name</th>
<th>Status</th>
</tr>
</thead>
<tbody>
{rows.map((row) => (
<tr key={row}>
<td>{row}</td>
<td>
<StatusBadge label="Active" />
</td>
</tr>
))}
</tbody>
</table>
);
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
export function StatusBadge({ label }: { label: string }) {
return <span className="badge">{label}</span>;
}
2 changes: 2 additions & 0 deletions eval/fixtures/field-patterns/app/components/ui/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
export { DataGrid } from "./DataGrid";
export * from "./StatusBadge";
12 changes: 12 additions & 0 deletions eval/fixtures/field-patterns/app/loadable.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import { Suspense, type ComponentType, type LazyExoticComponent } from "react";

/** The field app's code-splitting helper: every page element goes through it. */
export function Loadable(Component: LazyExoticComponent<ComponentType>) {
return function LoadableWrapper() {
return (
<Suspense fallback={<div>Loading…</div>}>
<Component />
</Suspense>
);
};
}
6 changes: 6 additions & 0 deletions eval/fixtures/field-patterns/app/main.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
import { createBrowserRouter } from "react-router-dom";

import { routes } from "./routes";

// The config is an imported identifier, not an inline array literal.
export const router = createBrowserRouter(routes);
15 changes: 15 additions & 0 deletions eval/fixtures/field-patterns/app/pages/HomePage.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import { useNavigate } from "react-router-dom";

export function HomePage() {
const navigate = useNavigate();
return (
<main>
<h1>Field Ops</h1>
<button type="button" onClick={() => navigate("/users")}>
View team
</button>
</main>
);
}

export default HomePage;
15 changes: 15 additions & 0 deletions eval/fixtures/field-patterns/app/pages/InvoicesPage.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import { Grid } from "../components";

import { useListInvoicesQuery } from "../store/api/invoicesApi";

export function InvoicesPage() {
const { data } = useListInvoicesQuery();
return (
<section>
<h1>Invoice history</h1>
<Grid rows={data ?? []} />
</section>
);
}

export default InvoicesPage;
15 changes: 15 additions & 0 deletions eval/fixtures/field-patterns/app/pages/UsersPage.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import { Grid } from "@ui";

import { useGetUsersQuery } from "@store/api/usersApi";

export function UsersPage() {
const { data } = useGetUsersQuery();
return (
<section>
<h1>Team directory</h1>
<Grid rows={data ?? []} />
</section>
);
}

export default UsersPage;
25 changes: 25 additions & 0 deletions eval/fixtures/field-patterns/app/routes.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
import { lazy } from "react";
import type { RouteObject } from "react-router-dom";

import { Loadable } from "./loadable";
import { HomePage } from "./pages/HomePage";

// Loadable(lazy(() => import())) page elements — the exact field pattern that
// produced 0 route nodes in the v0.3.0 validation run.
const UsersPage = Loadable(lazy(() => import("./pages/UsersPage")));
const InvoicesPage = Loadable(lazy(() => import("./pages/InvoicesPage")));

// Route arrays composed across variables and spread together, not written
// inline at the createBrowserRouter call site.
const billingRoutes: RouteObject[] = [{ path: "invoices", element: <InvoicesPage /> }];

export const routes: RouteObject[] = [
{
path: "/",
children: [
{ index: true, element: <HomePage /> },
{ path: "users", element: <UsersPage /> },
...billingRoutes,
],
},
];
7 changes: 7 additions & 0 deletions eval/fixtures/field-patterns/app/store/api/baseApi.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import { createApi, fetchBaseQuery } from "@reduxjs/toolkit/query/react";

export const baseApi = createApi({
reducerPath: "api",
baseQuery: fetchBaseQuery({ baseUrl: "/api" }),
endpoints: () => ({}),
});
14 changes: 14 additions & 0 deletions eval/fixtures/field-patterns/app/store/api/invoicesApi.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import { baseApi } from "./baseApi";

export const invoicesApi = baseApi.injectEndpoints({
endpoints: (builder) => ({
listInvoices: builder.query<string[], void>({
query: () => ({ url: "/invoices" }),
}),
payInvoice: builder.mutation<void, string>({
query: (id) => ({ url: `/invoices/${id}/pay`, method: "POST" }),
}),
}),
});

export const { useListInvoicesQuery, usePayInvoiceMutation } = invoicesApi;
11 changes: 11 additions & 0 deletions eval/fixtures/field-patterns/app/store/api/usersApi.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
import { baseApi } from "./baseApi";

export const usersApi = baseApi.injectEndpoints({
endpoints: (builder) => ({
getUsers: builder.query<string[], void>({
query: () => "/users",
}),
}),
});

export const { useGetUsersQuery } = usersApi;
7 changes: 7 additions & 0 deletions eval/fixtures/field-patterns/app/test-utils.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import { render } from "@testing-library/react";
import type { ReactElement } from "react";

/** The app's custom render: every test goes through this providers wrapper. */
export function renderWithProviders(ui: ReactElement) {
return render(<div className="providers">{ui}</div>);
}
10 changes: 10 additions & 0 deletions eval/fixtures/field-patterns/app/tsconfig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"compilerOptions": {
"jsx": "react-jsx",
"baseUrl": ".",
"paths": {
"@ui": ["components/index.ts"],
"@store/*": ["store/*"]
}
}
}
87 changes: 87 additions & 0 deletions eval/fixtures/field-patterns/golden.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
{
"failureMode": "D2",
"note": "Field-pattern regression fixture (6F.2): mirrors the shapes the 2026-07-15 ih-frontend validation run used where every gate scored 1.000 but the field run failed — tsconfig-path-alias barrel imports (@ui), Loadable(lazy(() => import())) page elements, route arrays composed in a separate file and passed to createBrowserRouter as an identifier, an RTK Query store (createApi/injectEndpoints/builder.query|mutation), and tests that render through a custom providers wrapper. Checks assert TARGET behavior; the ones the current scanner cannot satisfy are marked expectedFail with the step that must enable them (6F.3 aliases, 6F.4 RTK Query, 6F.5 object-config routes). xfail semantics remove stale marks automatically: once a step lands, its checks flip to unexpected-pass and gate until the marks are deleted. Verified working already in this shape: relative multi-hop barrel with rename (InvoicesPage's Grid), and covered-by detection through the custom render wrapper.",
"expect": {
"components": [
{ "name": "HomePage", "instances": 0 },
{ "name": "UsersPage", "instances": 0 },
{ "name": "InvoicesPage", "instances": 0 },
{
"name": "DataGrid",
"instances": 2,
"expectedFail": "enabled by 6F.3 — the @ui tsconfig-path alias import leaves UsersPage's <Grid/> unresolved (no instance node at all today)"
},
{ "name": "StatusBadge", "instances": 1 }
],
"queries": [
{ "terms": ["Team directory"], "status": "ok", "top": "UsersPage" },
{ "terms": ["Invoice history"], "status": "ok", "top": "InvoicesPage" }
],
"routes": [
{
"path": "/",
"component": "HomePage",
"layout": null,
"guards": [],
"expectedFail": "enabled by 6F.5 — createBrowserRouter(routes) with an imported identifier produces 0 route nodes today"
},
{
"path": "/users",
"component": "UsersPage",
"layout": null,
"guards": [],
"expectedFail": "enabled by 6F.5 — routes-to must survive the Loadable(lazy(() => import())) wrapper"
},
{
"path": "/invoices",
"component": "InvoicesPage",
"layout": null,
"guards": [],
"expectedFail": "enabled by 6F.5 — route comes from a spread of a separately-declared array"
}
],
"effects": [
{
"component": "HomePage",
"event": "onClick",
"effect": "navigates-to",
"to": "/users",
"expectedFail": "enabled by 6F.5 — navigates-to needs the /users route node to exist"
}
],
"journeys": [
{
"start": "/",
"expect": [{ "pages": ["/", "/users"], "end": "terminal" }],
"expectedFail": "enabled by 6F.5 — journeys from a path need route nodes (field regression: declined not-found)"
}
],
"attributions": [
{
"component": "UsersPage",
"endpoints": ["/api/users"],
"expectedFail": "enabled by 6F.4 — createApi/injectEndpoints/builder.query are unrecognized; 0 data-source nodes today"
},
{
"component": "InvoicesPage",
"endpoints": ["/api/invoices"],
"expectedFail": "enabled by 6F.4 — injectEndpoints with an object-form query ({ url }) in a second api slice file"
}
],
"blast": [
{
"node": "DataGrid",
"expect": [{ "node": "UsersPage" }],
"expectedFail": "enabled by 6F.3 — the shared-component blast radius misses UsersPage while its Grid instance is unresolved (field regression: 0 dependents)"
},
{
"node": "DataGrid",
"expect": [{ "node": "InvoicesPage" }]
}
],
"coverage": [
{ "component": "UsersPage", "tests": ["UsersPage.test"] },
{ "component": "StatusBadge", "untested": true }
]
}
}
15 changes: 11 additions & 4 deletions eval/src/checks.ts
Original file line number Diff line number Diff line change
Expand Up @@ -66,18 +66,25 @@ export function runChecks(
for (const expected of golden.expect.attributions ?? []) {
const id = `attribution:${expected.component}${expected.instanceAt !== undefined ? `@${expected.instanceAt}` : ""}`;
const found = traceEndpoints(graph, expected.component, expected.instanceAt);
// xfail-marked attributions stay out of the precision/recall tallies: a
// known-missing capability must not depress the global lineage metrics
// while marked. The unexpected-pass gate forces the marker off the moment
// the capability lands, which restores tally counting.
const tally = expected.expectedFail === undefined;
if (found === null) {
attribution.falseNegatives += expected.endpoints.length;
if (tally) attribution.falseNegatives += expected.endpoints.length;
finalize("attributions", id, false, expected.expectedFail, "trace target not found in graph");
continue;
}
const want = new Set(expected.endpoints);
const got = new Set(found);
const missing = [...want].filter((e) => !got.has(e));
const extra = [...got].filter((e) => !want.has(e));
attribution.truePositives += want.size - missing.length;
attribution.falseNegatives += missing.length;
attribution.falsePositives += extra.length;
if (tally) {
attribution.truePositives += want.size - missing.length;
attribution.falseNegatives += missing.length;
attribution.falsePositives += extra.length;
}
finalize(
"attributions",
id,
Expand Down
Loading