fix(sync): an import links the file it names when that file appears later

[colbymchenry]

Summary

codegraph sync (and the watcher) now links an import to the file it names when that file is added after the importer was indexed, or comes back after being deleted. Before, the import stayed unlinked until the importing file itself changed, or a full index ran. This is a pre-existing gap: #2388 changed which edge a Dart import got, not whether sync retries it.

It never relinked, for two reasons:

1. The retry key never matched. A failed ref is parked under name_tail, its last ./: segment (Incremental sync misses cross-file relations after a new export is added #1240), and a sync retries the failed refs whose tail is a node name in the changed files. For a path import that tail was the extension or a path fragment, and no node is ever named that:

   • package:app/b.dart → dart
   • inc/db.php → php
   • lib/foo.rb → rb
   • b.hrl → hrl
   • require('./req') → /req

2. Most languages never parked the import at all. An import that names no file reached the importing file's own import node through matchByQualifiedName, since that node's qualified name is the module path, and resolved there at 0.95. The ref was consumed, so nothing ever retried it. This covered:

   • TS/JS ESM ./x
   • Python import pkg.mod
   • C/C++/ObjC #include "a/b.h" and <stdio.h>
   • Lua require("lib.mod")
   • R source("R/x.R")
   • Solidity, Svelte and Vue imports
   • Go module paths
   • a C# using of an outside namespace

   fix(dart): an import or export links the file its URI names, not a namesake #2388 removed this for Dart only. A Lua/Luau require also bound by exact name to the local it is assigned to (local lfs = require "lfs" → variable lfs).

How:

• gateTargetKind (the resolveOne seam): an imports ref never resolves to an import node in its own file, so it parks as failed instead. The C# branch of matchByQualifiedName still returns the own using to end the lookup there, and the gate drops it.

• Lua/Luau require joins the no-fallthrough guard (PHP includes, COBOL, Nix, Dart, Terraform): it resolves to a module file or nothing.

• Path-aware tail (src/db/reference-tail.ts, moved out of queries.ts so the migration shares it). An imports ref with a / in it is parked under the last segment's stem:

  • package:app/b.dart → b
  • ./foo, ./foo.js → foo
  • inc/db.php → db
  • example.com/app/pkg → pkg

  Names without a / keep their old tail (pkg.mod → mod, com.z → z, App\Models::User → User), so the existing symbol retry of binding refs is unchanged.

• A second sync lookup, getRetryableFailedImports, runs after the existing symbol-name retry (results deduped by row id):

  • Keys from each ADDED file (importPathKeys): its name and stem, and its folder's name and stem. lib/src/d.dart gives src, d.dart and d. The folder key covers ./ui → ui/index.ts, pkg → pkg/__init__.py or pkg/init.lua, and Go packages. A key matches a failed import's tail or its whole name; the whole-name match covers bare #include "b.h", require 'db.php' and Dart part 'a.g.dart'. Modified files add no path keys, because they already existed.
  • The changed files' namespace/module node names, matched against a failed import's whole name when its tail differs: C# using Foo.Bar → the namespace node Foo.Bar, and Java import com.z.*. All node names would be wrong here: getNodeNamesByFiles includes the file's own import nodes (compress/gzip, @excalidraw/common), and that re-selected every other file's import of the same module. That cost up to 2,643 refs per added file on prometheus before I narrowed it.
  • Same 500-per-key ceiling as the existing lookup.

• Schema v12: two partial indexes over failed imports, (reference_kind, name_tail) and (reference_kind, reference_name).

  • Without them the lookup cost 11.5 ms per added file on vscode, and 83 ms for one query over common keys (index, types). The planner picks idx_unresolved_failed_tail / idx_unresolved_name: sqlite_stat1 averages 3 rows per tail across 828k failed refs, but common tails are heavily skewed, and every matching row is read just to check reference_kind.
  • With the leading reference_kind, the planner chooses them with or without stats: under 0.2 ms per query and 0.77 ms per added file.
  • The migration also rewrites the tails of path imports parked by an older version (dart → b), so those retry without a re-index.

Confirming the shapes in the issue

Import	On main
Dart package:app/b.dart	Parked, tail dart. Confirmed.
JS/TS ./foo	Tail /foo only for a CommonJS require('./foo'). ESM import … from './foo' never parks: it binds to its own import node.
C/C++ a/b.h	Never parks: own import node. A directory-less #include "c.h" did relink on main, through the CG-33 rebind, because the own import node is named c.h like the new file. a/b.h never did.
PHP inc/db.php	Parked, tail php. Confirmed.
Also missing on main	Ruby (rb), Erlang (hrl), Nix (nix / /bar), COBOL COPY X (tail fine, but no node is named X), Python (own node, or a binding tail no node has), Lua, Luau, R, Solidity, ObjC, Svelte, Vue, Java import com.z.*.
Already worked	Kotlin, Scala, Swift, Pascal, Rust use (symbol tails); C# (CG-33 rebind of the own using). Go package imports never link a file in either arm.
Still missing	Terraform module.<name>:file: the ref carries no path (the resolver reads source), so no key can reach it.

Validation

Main @837a1866 vs this change, indexed on the wasm path. Rebased onto #2391, which only adds Dart part refs; those go through the no-fallthrough Dart branch and never self-bind. The per-language probe and the new tests give identical results with the kernel loaded and with CODEGRAPH_KERNEL=0.

Repo	Files	Edges main → fix	Removed: own import statement	Removed: Lua require → its own local	Added	Parked as failed at the same site
redis @7f44037 (C)	898	78,366 → 77,087	1,279 (C 1,262, C++ 12, Python 5)	–	0	1,279 / 1,279
django @8c34887	3,020	192,379 → 191,035	1,344 (Python)	–	0	1,344 / 1,344
riverpod @4ba1be2	1,521	84,097 → 83,928	169 (TS/TSX/JS in the website docs)	–	0	169 / 169
excalidraw @ed10ac7	717	57,276 → 56,940	336 (TS 236, TSX 98, JS 2)	–	0	336 / 336
vscode @573eff8e	15,358	1,984,356 → 1,982,775	1,581 (TS 1,536, C++ 22, Python 6, C# 5, ObjC 4, JS 3, TSX 2, C 2, Java 1)	–	0	1,581 / 1,581
kong @8927af6 (Lua)	1,881	56,910 → 56,153	665 (Lua 656, Python 7, Go 2)	92	0	665 / 665
roact @1676d95	98	3,120 → 3,107	2	11 (Roact, TestEZ)	0	2 / 2
serilog @bebc771 (C#)	219	6,753 → 6,728	25	–	0	25 / 25
prometheus @39c878f (Go)	1,381	99,852 → 96,401	3,451 (Go 3,401, TSX 48, JS 2)	–	0	3,451 / 3,451

• Every removed edge is same-file. Nodes are identical everywhere, and nothing was added.
• Dead-code claims are unchanged on all 8 repos checked (not vscode). The unresolvedName exclusion count moves by at most 2.
• No cross-file edge changes, so file dependents and codegraph affected are unchanged on a full index.

Retry volume. This counts the extra failed refs the new lookup selects per added file, beyond what the existing symbol retry already selects. Each indexed file is treated in turn as the one a sync just added, using the fix-arm index:

Repo	Failed imports	Existing symbol retry (mean / p50 / p99 / max)	Added by this lookup (mean / p50 / p99 / max)	Retry every failed import on any add
redis	1,545	36 / 0 / 438 / 2,906	2 / 0 / 36 / 85	620
django	4,962	135 / 3 / 1,320 / 5,979	42 / 0 / 395 / 538	4,962
riverpod	2,064	114 / 17 / 808 / 1,780	16 / 0 / 109 / 130	2,064
excalidraw	3,176	149 / 15 / 677 / 1,521	6 / 0 / 163 / 409	3,176
kong	879	128 / 2 / 1,263 / 2,724	0.6 / 0 / 16 / 24	787
serilog	49	27 / 1 / 302 / 354	0 / 0 / 1 / 1	49
prometheus	8,980	512 / 190 / 2,978 / 3,835	20 / 3 / 197 / 328	8,980
vscode	27,191	280 / 85 / 2,266 / 20,849	20 / 5 / 291 / 465	19,644

The other option, retrying every failed import whenever a sync adds files, would cost 10–1,000× more per add. It also needs the path tails anyway: with the old tails every Dart import shares the key dart and every C include h, so each such group exceeds the ceiling and is skipped whole.

End to end on vscode. On copies of both arms' indexes I deleted chatService.ts (1,549 incoming imports edges from 399 files), ran codegraph sync, restored the file, and ran codegraph sync again:

• main: 925 edges from 269 files after the restore. Retry step: 8,339 refs, 6.8 s.
• fix: 1,319 edges from all 399 files. Retry step: 8,843 refs (+504), 8.1 s on a busy box.
• The fix brings back exactly the 394 file → file imports edges main loses.
• Still missing in both arms: 230 imports + 165 references + 2 implements edges, all to IChatService. 517 failed refs share that tail, which is over the existing per-name ceiling. Unchanged by this PR.

Tests

• New __tests__/sync-import-retry.test.ts (27 tests):
  • one sync case per affected language: Dart (incl. a generated part 'a.g.dart' written after the library was indexed), TypeScript, JavaScript (ESM + CommonJS), C, C++, Objective-C, PHP, Ruby, Python, Lua, Luau, Nix, R, Solidity, Erlang, COBOL, Java package wildcard, C# using, Svelte and Vue;
  • each case checks: nothing parked on its own statement, linked after sync() with the importer unchanged, and the result equal to a full re-index;
  • a delete/restore case;
  • importPathKeys shapes, the lookup's matching and ceiling, the v12 migration (tail rewrite + replay), and an EXPLAIN QUERY PLAN guard that the lookups use the new indexes.
• All 21 behavioral cases fail on main @837a1866:
  • Dart, PHP, Ruby, Luau, Nix, Erlang, COBOL and Java fail at the sync assertion;
  • the self-bound languages fail at the own-statement check. Their sync miss on main shows in the per-language probe, except C#, which main relinks through the CG-33 rebind; that case is a regression guard.
• tsc --noEmit is clean.
• Full suite on the rebased branch with the kernel loaded: 6,278 passed and 5 timed out, all in git-index-currency and sync, while another job loaded the box. Both files pass alone (71/71).
  • An earlier run of this change before the rebase: 6,269 passed and 4 failed (git-index-currency timeouts and a Windows EBUSY teardown in daemon-pid-reuse), and those files pass alone (14/14).
  • The ui project passes (18/18).

Not covered

• Terraform module refs (module.<name>:file) carry no path, so the retry can't key them.
• The 500-per-key ceiling applies as in the existing retry. Restoring a file that more than 500 failed imports name is not relinked by this lookup.
• Existing indexes keep their own-statement edges until the importing file is re-indexed or the project is; only then are those imports parked and retried. Parked path imports are rewritten by the migration. I did not bump EXTRACTION_VERSION: none of fix(dart): link a chained call only through its receiver's declared types (#750) #2376–fix(dart): a library links the files its part directives name #2391 did, and whether this warrants the re-index hint is your call.
• A change to config that makes an import resolvable (pubspec.yaml, tsconfig paths, go.mod, package.json workspaces) doesn't trigger the retry, because those aren't the added source file. A workspace package (@scope/pkg → packages/pkg/src/index.ts) only matches when its entry sits in a folder named for the package.
• Go package imports still never resolve to a file. They now park as failed instead of on their own statement, so:
  • adding a file to a package retries its importers for nothing (mean 20, p99 197 refs per added file on prometheus);
  • the viewer's file screen (unreleased) lists a Go file's own-module imports among its outside imports.
• COBOL copybook and Pascal unit names are matched case-sensitively.

🤖 Generated with Claude Code