Skip to content

fix(sync): imports through a re-exporting module follow it when it is deleted or added - #2512

Merged
colbymchenry merged 1 commit into
mainfrom
claude/sync-deleted-reexport
Oct 12, 2026
Merged

colbymchenry merged 1 commit into
mainfrom
claude/sync-deleted-reexport

Conversation

@colbymchenry

Copy link
Copy Markdown
Owner

Problem

codegraph sync and the watcher never revisited the files that import through a module when that module was deleted or added. Here "through" means a module that re-exports what it imports: a package's __init__.py, an index.ts barrel, or a plain module that re-exports. A fresh index of the same tree disagreed in both directions:

Found while working on #2507: its restore-after-delete test couldn't pass, because deleting the barrel unlinked nothing.

Fix

src/resolution/module-importers.ts: when sync removes or adds files, it re-opens the references that the files importing them make through those imports, back to pending for the sync's sweep. The sweep binds them against the post-sync graph as a full index does: through the module when it is there, by name or not at all when it isn't. A failed one is parked under its module's key (#2501, #2507), so it relinks when the module comes back.

  • Removal (in the orchestrator, once for all the removed files, before any is deleted): every resolution edge of an importer whose refName is a binding of a statement naming a removed file, or a member read through one (Order.objects, ns.fn). Edges into a removed file itself are left to the existing resurrection.
  • Addition (after the Incremental sync misses cross-file relations after a new export is added #1240 retry, before the rebind and sweep): the same importers' edges from before this sync's resolution (getMaxEdgeId mark), their failed references through those bindings, and the failed import of the module itself.
  • Finding the importers (for all the changed files at once):
    • the imports edge into the file's node, for ES modules and dotted Python imports;
    • for Python, import nodes whose module path denotes the file, relative or absolute (from . import Order, from .. import Order, from shop import Order; the relative ones never resolve to a file);
    • or import nodes that name the package around the file and import the file's module from it (from django.db import models, then models.Model);
    • an import inside a function has no import node, so it is found through what the modules pass on: a Python file with edges into the targets of the changed modules' own import edges counts when its imports name one;
    • a Python module that re-exports a changed one leads on to its own importers: a star import, or a package's __init__.py importing names from it. Only the names it passes on are followed, three modules deep, as far as the resolver's pythonModuleSymbol chases a name.
  • Scope: only Python and the ES-module family (TS, TSX, JS, JSX, ArkTS, Vue, Svelte, Astro), whose import bindings are what references are written as. An #include or a Go package import binds no reference name of its own.
  • Cost: importer edges and failed references are filtered by the bound names in SQL (THROUGH_NAMES). Each importer is visited once per sync, and each file is read once for its import statements.

No schema change and no extraction change (so no kernel change). Fresh indexes are byte-identical to main's.

Validation

Tests. __tests__/sync-module-importers.test.ts has 6 tests: a Python package and module, and an ES-module barrel, each deleted, deleted and restored, and added after its importers were indexed. Each compares the synced graph with a fresh index of the final tree in a second folder.

  • The Python fixture covers an absolute import, imports from inside and beside the package, a renamed re-export, a plain re-exporting module, from app import orm with orm.Model, a function-local import, a star-importing package and a package re-exporting one name, plus 501 failed imports named shop (over the retry's ceiling).
  • On main, 5 of 6 fail. The one that passes, the ES-module barrel deleted and restored, only passes because main leaves the stale edges in place and they happen to be correct again once the barrel is back.
  • Each discovery path is load-bearing: removing it fails 1 to 4 of the tests. The paths are the import nodes, the package match, what the modules pass on, the member read in SQL, the module's own import, the add side, star imports, __init__ re-exports and the nested levels.
  • They pass with CODEGRAPH_KERNEL=0 too.
  • Related suites (sync, watcher, Python, imports): 27 files passed and 1 skipped; 397 tests passed and 9 skipped.
  • Full suite on Windows: 547 files passed and 6 skipped; 6,938 tests passed, 0 failed, 82 skipped.

A/B on real repos. Each arm's synced graph is compared with its own fresh index of the same tree in another folder. Cells are the differing graph lines, main → this PR.

  • barrels: every 2nd non-empty __init__.py or index.{ts,tsx,js,jsx} (django 36, netbox 27, pydantic 2, fastapi 2, vben 114, vue3-element-admin 12, takenote 5).
  • every 7th: every 7th source file (django 434, netbox 192, pydantic 65, fastapi 167, vben 209, vue3-element-admin 39, takenote 17).
  • deleted: index the repo, delete them, sync.
  • restored: delete them, sync, put them back, sync.
  • added late: index without them, put them back, sync.
repo barrels deleted every 7th deleted barrels restored every 7th restored barrels added late every 7th added late
django 58,456 → 1,336 16,314 → 834 2,150 → 6 1,584 → 152 46,152 → 1,334 7,966 → 232
netbox 10,246 → 244 5,368 → 330 1,100 → 224 4,796 → 4,736 9,750 → 462 7,166 → 4,746
pydantic 13,966 → 24 406 → 394 306 → 158 7,512 → 762 8,450 → 182 7,734 → 982
fastapi 452 → 0 4,090 → 16 0 → 0 36 → 0 270 → 0 2,140 → 16
vben 642 → 560 145 → 145 0 → 0 70 → 64 560 → 478 116 → 110
vue3-element-admin 518 → 0 182 → 8 0 → 0 8 → 0 518 → 0 182 → 0
takenote 16 → 16 12 → 12 0 → 0 0 → 0 0 → 0 0 → 0
  • requests, django-realworld and ts-express: 0 → 0 in every scenario.
  • Controls (gin in Go, serilog in C#, kong in Lua; every 7th file deleted, restored and added late): the synced graphs are byte-identical to main's in all 9 runs.
  • Fresh indexes: byte-identical to main's on all 13 repos.
  • By class, against main, every class of difference went down except one function reference, async_view, in Django's two every-7th restore and add scenarios. It is an existing difference between pooled and sequential resolution: this PR re-opens the reference, and the sweep's small batch resolves it sequentially, where a fresh index resolves it pooled.

Cost. Sync wall time, main and this PR, interleaved over two rounds. The machine was shared, so a few runs are noisy (fastapi's main at 15.2 s).

scenario main this PR
django, 36 __init__.py deleted 10.4 / 10.5 s 17.0 / 17.2 s
netbox, 27 __init__.py deleted 4.5 / 4.4 s 6.1 / 6.3 s
django, 434 files deleted 17.9 / 18.2 s 21.1 / 22.1 s
pydantic, 2 __init__.py deleted 2.1 / 2.4 s 3.3 / 4.0 s
vben, 114 index.ts deleted 3.8 / 6.0 s 4.0 / 5.2 s
netbox, 192 files added late 11.4 / 14.3 s 13.2 / 13.1 s
fastapi, 167 files added late 6.1 / 15.2 s 6.7 / 8.3 s
vue3-element-admin, 12 index.ts added late 2.1 / 2.5 s 2.2 / 2.9 s
requests, 5 files deleted 1.0 / 1.1 s 1.1 / 1.1 s

In Django's 36-package deletion, the importer pass re-opens 29,501 references in 3.1 s: it finds the importers, then swaps their edges for references. The rest of the difference is the sweep re-resolving them, which a fresh index does too. For scale, a fresh index takes about 31 s for Django and 15 s for netbox. An earlier draft of this PR, which visited the importers once per removed file and filtered their edges in JS, took 6–11 s for the same pass.

Residues

  • Nested ES-module barrels (vben's 560 and 478): an index.ts that re-exports another barrel (export * from './dropdown-menu') is not followed. export … from leaves no import edge or node. export * and the specifier leave nothing, and a named re-export leaves only an imports reference for its original name. So an outer barrel can't be found from the index. It would take an extraction change (edges for export … from, which changes every fresh index) or a directory scan.
  • Names past the Incremental sync misses cross-file relations after a new export is added #1240 ceiling (netbox's every-7th 4,736 and 4,746): from django.test import TestCase names a package outside the repo, so a fresh index links TestCase by name to netbox's own test class. The same goes for assertHttpStatus (801) and add_permissions (576). When the defining file comes back, the retry looks these up by name, and each has more than 500 failed references waiting. This is the same on main and is deliberate (codegraph index wedges indefinitely at "Resolving refs" 94% on a repo with a large *tracked* JS/TS asset dir and there's no way to exclude it #999's rationale).
  • Calls typed through a base class from the deleted package (most of Django's barrel residue: 481 pairs when deleted, 445 failed calls when added late): obj.save() or instance.refresh_from_db() on a test model whose class R(models.Model) base came through django/db/models/__init__.py. The extends models.Model reference is re-opened and fails, as in a fresh index. The calls name a local variable, not an import, so they keep their edges. When the package is added late, the same calls stay failed, because save is past the Incremental sync misses cross-file relations after a new export is added #1240 ceiling. Both are the same on main.
  • takenote's 16 and 12: an Express mount's route names after router/index.ts is deleted, and NoteItem. Both are the same on main.
  • pydantic's 309 failed calls in its every-7th restore and added-late runs: the same 309 on main. A Python module that re-exports by name without being a package isn't followed past the first level (only __init__.py files and star imports are).

Overlap

🤖 Generated with Claude Code

… deleted or added

Deleting a module that re-exports what it imports (a package's
`__init__.py`, an `index.ts` barrel) left every file importing through it
linked to the symbols it re-exported: those edges point into files that
stay, so the removal cascaded none of them. When such a module was added or
came back, its importers kept the edges a name lookup had made in its
absence, or stayed failed; and in a large Python project the failed import
of the module itself stayed failed, because the failed-import retry skips a
stem with more than 500 imports waiting (Django's `test`, `models`).

Sync now re-opens, as pending references for its sweep, what the files
importing a removed or added module bind through those imports: before the
removed files are deleted, and after the retries for the added ones (with
their failed references and the module's own failed import). Importers are
found by the import edge into the file, by Python import nodes naming the
module or its package, and through what the module passes on (an import
inside a function has no node); in Python, a package or star import
re-exporting a changed module leads on to its own importers, as deep as the
resolver chases a name. All changed files are handled in one pass, with
importer edges and failed references filtered by the bound names in SQL.

Python and the ES-module family only; no schema or extraction change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@colbymchenry
colbymchenry merged commit 7ba8eee into main Oct 12, 2026
@colbymchenry
colbymchenry deleted the claude/sync-deleted-reexport branch October 12, 2026 07:10
colbymchenry added a commit that referenced this pull request Oct 14, 2026
…2560)

A lazy React Router route whose module is a barrel, and whose page sits in a
folder below the barrel's, stayed unlinked after `codegraph sync` added the
page or gave it its export, though a fresh index linked it. In
DigiShop.Frontend, `@/pages` is `src/pages/index.ts`, which imports `./Home`,
`src/pages/Home/index.tsx`, and exports it again. While the page is missing,
the route's reference fails and waits under the key of the module it names,
`module:pages` (#2422). The retry looks failed references up by the keys of
the changed files, and a file in `pages/Home/` gives only `module:Home` and
its own stem. The limit, which the `export *` barrels of #2436 share, was
recorded with #2558.

After the failed-ref retry, which makes the barrel's import edge into an
added file, sync now walks up the import edges from each file it added, or
gave a definition, to the barrels that pass it on: `export … from`, or an
import exported again (`import { X } from './x'; export { X }`, `export
default X`), as deep as the resolver chases a name. It re-opens, for the
sweep, the failed references waiting under those barrels' keys.

Only keys a failed reference waits under are looked up, in one DISTINCT
over the failed rows parked under module keys, so a sync with none reads
nothing, and a body-only edit seeds nothing. The ImportReader that the
added-files reopen (#2512) uses is shared, and gains exportedBindings,
behind a regex pre-check, since most importers export nothing.
defaultExportBinding moves to alias-binding.ts for this walk and for
react.ts's exportsBinding.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant