Skip to content

fix(go): resolve unexported receivers and nested modules within the right package (#2323, #2322) - #2361

Merged
colbymchenry merged 2 commits into
mainfrom
fix/2323-2322-go-package-resolution
Oct 5, 2026
Merged

colbymchenry merged 2 commits into
mainfrom
fix/2323-2322-go-package-resolution

Conversation

@colbymchenry

Copy link
Copy Markdown
Owner

Summary

Two Go resolution gaps, fixed together because they turned out to be one problem: knowing which package a Go name belongs to.

Why one PR: each fix alone sends calls to the wrong package, and the machinery that prevents that is shared:

  • Typing unexported receivers alone (the fix(go): resolve field-chain calls on unexported method receivers #2325 pattern on main) sent 1,042 of harbor's 1,998 new calls to another package's same-named type, because the method lookup only preferred the caller's file: 653 a.SendError() calls in harbor's handlers went to common/api's BaseAPI instead of the handler package's own embedded BaseAPI, and d.dao.Create() in every daoTestSuite went to the first indexed DAO::Create.
  • Reading nested modules alone (fix(go): resolve calls inside a Go module whose go.mod is below the project root (#2322) #2324 on main) makes those modules' package-qualified field types and type references resolvable by name: on harbor that added 47 self-loops (m.delegator.Count() in a caching wrapper resolving to the wrapper itself instead of artifact.Manager), and on etcd about 180 type references landing on same-named methods (testutil.TB → LazyCluster::TB, mvcc.KV → KVGetter::KV).

Both need "which directory declares this type", which needs the module map from #2322 for qualified names. Splitting would either duplicate that logic in both PRs or ship one of them with known wrong edges.

Cause

  1. localReceiverTypePatterns (Go) only accepted PascalCase types for parameters and receivers, so s in func (s *server) had no type and matchGoFieldChainCall / the typed method path dropped the call (Go: calls through a struct field are not resolved when the receiver type is unexported #2323).
  2. resolveMethodOnType and matchGoFieldChainCall look types and methods up by name across the whole project, breaking ties by call-site file. Go type names are only unique per package (directory), so server, DAO, Manager, Controller collide constantly; the name-based supertype walk mixed every same-named type's embeddings.
  3. loadGoModule read only <root>/go.mod; getGoModule() was the single source for "is this import in the project" and "which directory is it" (Go: cross-package calls are not resolved when go.mod is not at the project root #2322).
  4. A name written through a package (job.OPCommand, storage.Querier, pb.RangeRequest) is indexed bare; only external qualifiers were checked, so an in-project qualifier didn't stop the name matcher from picking a same-named symbol in another package (often a method, or the caller itself).

Fix

  • Receivers and parameters of unexported types (Go: calls through a struct field are not resolved when the receiver type is unexported #2323). One Go pattern accepts a lowercase type only where a parameter list puts it — after ( / , / line start and before , / ) / line end — which covers func (s *server), func (c *cache[T]), func f(h *handler) and var ( … ) block lines, while a lowercase ident type pair elsewhere still isn't read as a declaration. This is @GoDiao's anchored-receiver approach from fix(go): resolve field-chain calls on unexported method receivers #2325, widened to parameters with the validation below.
  • Package-scoped method resolution. The receiver inference now also reports the type as written (server vs store.Manager). For Go, resolveMethodOnType takes the declaring package's directory: the caller's package for a bare name, the imported package's directory for a qualified one. Only that package's Type::Method counts; a method the type lacks is looked up on the types its declaration embeds (read from the declaration, so type EphemeralKV struct{ RemoteKV }, *cached.BaseManager and interface embedding like KV { ReadView; WriteView } all work), never on another package's same-named type. When the package declares no such type (dot import, unindexed file) resolution falls back to the old name-based behaviour.
  • Field chains. matchGoFieldChainCall looks for the base struct in its declaring package first and resolves the field's type in the struct's own package (bare) or the imported package (qualified).
  • Nested and sibling modules (Go: cross-package calls are not resolved when go.mod is not at the project root #2322). @danusha2345's discovery from fix(go): resolve calls inside a Go module whose go.mod is below the project root (#2322) #2324: the resolver reads the nearest go.mod at or above every directory with an indexed .go file (memoized per directory, no disk walk, never above the project root; ignored and vendored trees add nothing) and maps an import path to the module with the longest matching module path, with the importing file's own module breaking a tie between two modules declaring the same path. A module under testdata/ or a _/. directory only serves its own files (the go tool ignores those directories; mattermost has a stub module there). go.work needs no separate reading: every module it lists inside the project owns indexed Go files and is found this way, and modules outside the project aren't indexed. The context API is now getGoPackageDir(importPath, fromFile?), used by isExternalImport, the cross-package call resolver, the external-qualifier guard and the field-chain matcher.
  • Names written through a package. A bare reference whose source spells a package qualifier now resolves through the import to the exported symbol in that package's directory (same lookup as pkg.Func() calls), and a name-matched candidate outside the qualifier's package is rejected. A call or method value landing on a method is exempt (a package holds no methods, so that qualifier is a variable shadowing the import — prometheus's labels := &maxHeap{}; labels.get()).
  • No EXTRACTION_VERSION bump: resolution only (the fix(go): resolve calls inside a Go module whose go.mod is below the project root (#2322) #2324 bump is dropped). No kernel change.

Verification

  • New __tests__/go-unexported-receiver.test.ts (3 tests) and __tests__/go-nested-module.test.ts (11 tests, adapted from fix(go): resolve calls inside a Go module whose go.mod is below the project root (#2322) #2324's file plus etcd's root-plus-sibling layout, qualified type references, a caching wrapper and a testdata module). With src/ reverted to main (CODEGRAPH_KERNEL=0): 12 of 14 fail; the two controls (an import that only shares a prefix with a module path stays unresolved; a root go.mod resolves as before) pass. With the fix: 14/14 pass.
  • The contributor PRs alone, against the same tests: fix(go): resolve field-chain calls on unexported method receivers #2325 passes the issue's case but sends s.Close(), s.SendError() and s.store.Put() to the other package's same-named server, BaseAPI and store; fix(go): resolve calls inside a Go module whose go.mod is below the project root (#2322) #2324 passes its own cases but links job.OPCommand to the method returning it, m.delegator.Count() to the wrapper itself, and lets a testdata module answer for the real one.
  • npx tsc --noEmit clean; npm run build OK.
  • The issues' own repros on the built CLI: server receiver → callers AddItem 0 → 1; svc/go.mod → callers New 0 → 1 and callers CreateItem 0 → 1; the root-go.mod and Server variants unchanged (1 / 1).
  • With the native kernel loaded (built from this tree, CODEGRAPH_KERNEL_PATH): the Go tests pass, and the etcd and harbor graphs below are byte-identical to the wasm-path ones in both arms.
  • Full suite (Windows, box at 100% CPU from other sessions): 5,669 passed, 53 failed in 25 files, every one Test timed out in 5000ms (plus the EBUSY cleanup that follows a timeout), no assertion failure. Rerun alone, 18 of those files passed at once; the other 7 (git-index-currency, laravel-route-paths, mcp-stale-refusal, mcp-status-freshness, mcp-unindexed, orphaned-refs-sweep, worktree-detection) timed out once more, then passed on the next run, as they do on main. After rebasing onto main (1b56cb1): tsc clean, build OK, the Go tests pass and etcd's graph is unchanged.

Validation (before = main 26e8488, whole-repo shallow clones, Go→Go edges keyed by source/target qualified name + file + line + column)

Repo (commit) Layout Go→Go edges Go→Go calls Added Removed (re-resolved / dropped)
etcd-io/etcd (6bb7e5e) root module + 14 sibling and nested modules (go.work) 52,530 → 66,284 18,074 → 25,265 +13,827 73 (60 / 13)
goharbor/harbor (721c6d4) only src/go.mod 62,423 → 77,615 16,308 → 25,236 +15,819 627 (615 / 12)
prometheus/prometheus (770ca8f) root module (+ small tool modules) 90,970 → 92,399 40,050 → 41,493 +2,249 820 (783 / 37)

No non-Go edge changed. New self-loops (etcd 5, harbor 2, prometheus 8) are all real recursion (c.Copy(...) inside client.Copy, x.left.height(...), retrying i.Next()).

Per fix, measured by switching one part off in the built branch (Go→Go edges added / removed by that part):

Part etcd harbor prometheus
Unexported receivers and parameters (#2323) +1,319 / −18 +3,568 / −176 +1,347 / −12
Nested and sibling modules (#2322) +12,973 / −17 +13,355 / −93 0 / 0 (no cross-module imports)
Package scoping and names through a package (shared) — — +902 / −808, 778 of the 808 re-resolved to the right package (prometheus is the one repo where neither part above adds much, so this row is that part alone)

The #2323 removals are name guesses (instance-method@0.65) now typed, plus field reads that used to pass as method values. For comparison, #2325 alone measured +1,131 on prometheus and +1,999 on harbor here, but 1,042 of harbor's were in the wrong package.

Added edges come from the typed path (0.9 receivers, 0.85 field chains) and from import resolution (0.9: pkg.Func() calls, pkg.Type{} literals and type references in sibling/nested modules). Spot-checked against the source, all correct (40+ across the three repos), e.g.:

  • etcd ap.authClient.RoleDelete() → api/etcdserverpb AuthClient::RoleDelete (field authClient pb.AuthClient, a sibling module); cr.dial(t) in func (cr *streamReader) run() → streamReader::dial; s.cls.MemberUpdate() through the one-line type cls2clc struct{ cls pb.ClusterServer }; betesting.NewTmpBackend(...), clientv3.NewKVFromKVClient(...); backend.Backend / testutil.Action type references.
  • harbor api.SendError(ctx, err) in every handler → the handler package's own embedded BaseAPI::SendError; m.delegator.NonEmptyRepos(ctx) → repository.Manager; d.artDAO.Create(...) → pkg/artifact/dao DAO::Create; suite.cache.Delete(...) → lib/cache Cache::Delete; a.client.Do(r) in func getJwtToken(a *adapter, …).
  • prometheus c.chunk.Bytes() through a named result (c *memChunk, …); q.stats.GetSpanTimer(...) in func (ng *Engine) exec(ctx, q *query); s.fh.Copy(); wctx.kv.Put(...); index.Postings / storage.Querier / promql.QueryEngine result types.

Removed edges (1,520): 1,458 are the same call site or reference now resolved to the right package's symbol; sampled ones were all wrong before: harbor's handlers calling common/api's BaseAPI::SendError, suite.dao.Count() landing on pkg/accessory/dao, model_tag.Tag{ on a method ImageRepository::Tag; prometheus's tsdb.Options on discovery/refresh's Options, parser.Parser on textparse's, index.Postings result types on the method declaring them, h.bytesPool.Get() on chunkenc's Pool instead of zeropool's; etcd's pb.LeaseKeepAliveResponse on the client's own type, cli.Get(...) on kubernetes.Client instead of the embedded clientv3.KV. The 62 dropped were wrong too: field reads taken for method values (s.walDir, r.addr, m.ID, m.t, s.F all used to link to unrelated functions or methods named like the field), compression.Type where the real target is a type Type = string alias that isn't indexed (it used to link to tsdb/compression's different Type, or to a method fSample::Type), and two calls to a method xds's SDConfig doesn't have (they used to land on aws.SDConfig).

Indexing time and CPU are unchanged (harbor, process CPU incl. workers: 69.6 s / 68.7 s before vs 68.6 s / 69.4 s after; interleaved wall-clock runs within noise on all three).

Credit

@GoDiao reported both issues with precise causes and measurements, and wrote #2325 (the anchored receiver pattern adopted here, widened to parameters). @danusha2345 wrote #2324, whose module discovery (loadGoModule(moduleDir), nearest-go.mod memo, longest-prefix lookup with the own-module tie-break) and test layout are adopted here.

Related open PRs that overlap: #1954 (Go receiver typing incl. unexported/qualified params, by @danusha2345) and #1521 (Go multi-module layouts, by @ericquan8).

🤖 Generated with Claude Code

colbymchenry and others added 2 commits October 5, 2026 16:06
…ight package (#2323, #2322)

Calls through a method receiver or parameter of an unexported type
(`func (s *server) Create() { s.service.AddItem() }`) had no edge: the
Go receiver inference only accepted PascalCase types (#2323). And only
the project-root go.mod was read, so a module in a subdirectory, or one
beside other modules as in etcd, treated imports of itself as
third-party (#2322).

Fixing either alone sends calls to the wrong package, because Go type
names are unique only per directory while types and methods were looked
up by name across the project. The receiver pattern alone sent 1,042 of
harbor's 1,998 new calls to another package's same-named type; reading
nested modules alone made harbor's caching wrappers call themselves and
put about 180 of etcd's type references on same-named methods. Both
share one fix:

- A lowercase receiver or parameter type is read only where a parameter
  list puts it (receivers, parameters, multi-line lists, var blocks).
- Receiver inference also reports the type as written. For Go,
  resolveMethodOnType takes the declaring package's directory (the
  caller's for a bare name, the import's for a qualified one) and counts
  only that package's methods, then the types its declaration embeds
  (struct and interface embedding, read from the source) instead of the
  name-based supertype union. The field-chain matcher scopes its struct
  and field types the same way.
- The resolver reads the nearest go.mod of every directory holding
  indexed Go files (no disk walk; a module under testdata/ or a _/.
  directory serves only its own files) and maps an import path to the
  module with the longest module path, the importing file's own module
  breaking a tie. getGoPackageDir replaces getGoModule.
- A bare name written through a package (`job.OPCommand`) resolves
  through the import into that package's directory, and a name-matched
  candidate outside the qualifier's package is rejected; a method reached
  through a variable that shadows an import is exempt.

Verification: new go-unexported-receiver (3) and go-nested-module (11)
tests: 12 fail on main (the 2 controls pass), all 14 pass. The issues'
repros go from 0 to 1 caller; the root-go.mod and `Server` variants are
unchanged.

Validation, main 26e8488 -> this branch, whole repos, Go->Go edges keyed
by source/target qualified name, file, line and column:
- etcd 6bb7e5e: 52,530 -> 66,284 (+13,827 / -73), calls 18,074 -> 25,265
- harbor 721c6d4: 62,423 -> 77,615 (+15,819 / -627), calls 16,308 -> 25,236
- prometheus 770ca8f: 90,970 -> 92,399 (+2,249 / -820), calls 40,050 -> 41,493
Of the 1,520 removed edges, 1,458 are the same call site or reference
now resolved to the right package's symbol; the other 62 were field
reads taken for method values, names whose real target is not indexed
(a `type T = string` alias), and a call to a method the receiver's type
does not have. Indexing time and CPU are unchanged.

Co-authored-by: Diao Shengjia <104132148+GoDiao@users.noreply.github.com>
Co-authored-by: danusha2345 <ewidusoc498@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@colbymchenry
colbymchenry merged commit deac771 into main Oct 5, 2026
colbymchenry added a commit that referenced this pull request Oct 5, 2026
… summary time covers the whole run (#2334) (#2362)

Indexing pretix took 2.2x as long since #2163 (issue #2334), and the
bundled minified d3 was what #2344 left of CPython's 1.6.2 slowdown. Two
JS/TS checks that first shipped in 1.6.2 redid whole-file work for every
reference:

- matchDestructuredCallResult (#2163) ran for every bare call in a file
  holding `const {` / `let {` / `var {`: per call it re-tested that
  pattern on the source, then stripped, blanked and brace-walked every
  line above the call and matched the binding regex over it.
- jsFunctionLocalScope (#2226) re-stripped the calling function's lines
  above each reference. In a minified script every function's text runs
  to the end of its one line, so each strip was the whole file.

CPU profiles, main -> this change: matchDestructuredCallResult 82.4 s
-> 1.7 s and jsFunctionLocalScope 12.3 s -> 5.2 s on pretix; 35.4 s ->
0.37 s and 23.3 s -> 1.0 s on CPython's Lib/profiling + Doc/_static.

Now:
- A file's `const { … } = f(…)` bindings are read once per resolution:
  the blanked code, each binding's end, callee and keys, the block it
  sits in and where that block closes, and the spans the blanker read as
  regex literals. A call looks up its own name; only a call through a
  destructured name scans the code between that binding and itself for
  shadowing, and only once the callee is found to return the key (a
  `require(…)` destructuring, for one, needs no scan).
- Blanking the text above a call on its own gives the file's blanked code
  up to the call: the stripper and blanker look past a character only for
  a comment opener's second character and for a regex literal's closing
  `/`. So the prefix's bindings are the file's that end at or before the
  call (a match never reads past its `(`, and one running past the call
  holds no `{`…`(` to start another), and its brace stack follows from
  where each block closes. For a call inside a span the file reads as a
  regex literal (a division, in minified code) the prefix is the code up
  to the `/` plus the short tail blanked on its own; where a binding
  statement could run across that `/` (inside its `{ … }` or `< … >`), and
  for out-of-range positions, the old per-call scan still runs.
- jsFunctionLocalScope strips a function's lines once and cuts at the
  reference's line end; functions that start on the same line share the
  regex answer by (file, first line, reference line, name).
- The new memos, and JS_FN_LOCAL_MEMO (never cleared before), drop with
  the other per-file memos in clearNameMatcherMemos.

The summary line printed orchestrator.indexAll's durationMs, which covers
extraction only, beside node and edge totals recomputed after resolution:
the issue's 19.9 s run printed "in 1.7s". CodeGraph.indexAll and sync now
report the whole run's wall time, which `codegraph init`, `index` and
`sync`, the MCP auto-sync log and the telemetry duration bucket read.

Verification: nodes, edges and unresolved_refs dumps keyed by
file|kind|qualified_name|start_line are byte-identical to origin/main
8998697 (with #2357's new destructuring and Vue template refs) on all
24 runs: pretix, mealie and CPython's Lib/profiling + Doc/_static, 8
each; and to deac771 (#2358, #2360, #2361) on one more round of each. A differential harness ran old and new matchDestructuredCallResult
on every call position of mealie (10,330) and the CPython assets (9,170,
742 inside misread regex literals) and on 16,000 random token-soup
sources (1.33M positions), and old and new jsFunctionLocalScope on ~500K
(function, line, name) triples: no difference; it reports one for each
deliberate mutation of the new code that is not provably equivalent.

Medians of 4 interleaved runs on a loaded 16-thread Windows box, process
CPU / wall / peak RSS: pretix 160.2 s / 75.9 s / 2.47 GB -> 88.7 s /
43.5 s / 1.93 GB; CPython assets 75.1 s / 37.6 s / 1.23 GB -> 32.1 s /
20.4 s / 0.81 GB; mealie (no bundles, the control) 39.2 s -> 38.6 s CPU.

__tests__/js-resolution-work.test.ts counts strips: a file with a
destructured binding and 40 bare calls is stripped for the check once
(41 on main), a 40-line function once (40 on main); each count fails when
only its own half of the fix is reverted. full-pipeline.test.ts requires
indexAll's and sync's durationMs to cover a linking pass delayed 250 ms
past the orchestrator's time (on main: 218 < 468).

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
danusha2345 pushed a commit to danusha2345/codegraph that referenced this pull request Oct 6, 2026
…thod

matchMethodCall's name-only Strategy 3 single-candidate branch matched
`other.m()` written inside `m()` back to `m` itself whenever `m` is the
only method of that name: Scala's `wrapped.unbind(v)` inside
WrappedMapping.unbind. An explicit receiver that names another object
can't be the enclosing method's own instance, so that self-loop is the one
guess known to be wrong — return no edge. (The receiver/class word-overlap
branch already refuses the caller itself, colbymchenry#2221.)

Real recursion is kept: `this.`/`self.`/`super.`/`cls.`/`Self::`
receivers and bare calls (never reach Strategy 3). Go's named method
receiver (`func (r *T) m()` calling `r.m()`) is resolved on its declared
type before Strategy 3 (colbymchenry#2361), so it needs no exemption here; the test
still pins it. Strategies 1/2 (explicit class-name receiver) are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
danusha2345 pushed a commit to danusha2345/codegraph that referenced this pull request Oct 6, 2026
…liases; no guess for outside types

On top of colbymchenry#2361, which types unexported receivers and parameters and
resolves a method in the package that declares the receiver's type, four
receiver shapes still got no type, and a receiver typed outside the
project still got a guess by method name:

- A variable named like a standard-library package (`ring`, `token`,
  `parser`). Every `ring.X()` was skipped as a call into that package,
  whether or not the file imports it. It is skipped now only when the file
  imports the package; otherwise the receiver is typed like any other, and
  one whose type is unknown still gets no edge.
- A parameter or receiver of a package-qualified type (`s *store.Store`).
- A receiver bound to a call: `r := newRing()`, `s, err :=
  store.NewStore()`, `if r := find(id); r != nil`, a call spread over
  several lines, a conversion (`list := model.BotList(bots)`). The type is
  the callee's first result as its signature spells it, read in the
  callee's file. Only when the call is the whole right-hand side, and not
  when it takes the receiver itself.
- An alias (`type Context = web.Context`, `= *web.Context`, grouped). An
  alias has no node, so it is read from the package's files and followed
  to the type it names, for a receiver and for an embedded field.

A receiver whose declared type comes from a package outside the project's
modules (`conn net.Conn`, `ctx context.Context`, `req *http.Request`, an
alias of one, the result of an outside function) gets no edge:
`ctx.Done()` went to the one project type declaring a `Done`. A project
function's result declared as an outside type by value
(`http.RoundTripper`) is left as it was, since that is usually an
interface a project type implements.

Resolver only: no extraction change, the receiver scan keeps its
per-line memo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
danusha2345 pushed a commit to danusha2345/codegraph that referenced this pull request Oct 7, 2026
…thod

matchMethodCall's name-only Strategy 3 single-candidate branch matched
`other.m()` written inside `m()` back to `m` itself whenever `m` is the
only method of that name: Scala's `wrapped.unbind(v)` inside
WrappedMapping.unbind. An explicit receiver that names another object
can't be the enclosing method's own instance, so that self-loop is the one
guess known to be wrong — return no edge. (The receiver/class word-overlap
branch already refuses the caller itself, colbymchenry#2221.)

Real recursion is kept: `this.`/`self.`/`super.`/`cls.`/`Self::`
receivers and bare calls (never reach Strategy 3). Go's named method
receiver (`func (r *T) m()` calling `r.m()`) is resolved on its declared
type before Strategy 3 (colbymchenry#2361), so it needs no exemption here; the test
still pins it. Strategies 1/2 (explicit class-name receiver) are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
colbymchenry added a commit that referenced this pull request Oct 7, 2026
…method (#2444)

Both extractors record `srv.(KVServer).Range(ctx, in)` as a bare `Range`
call at the column where its receiver expression starts, as they record
any call through an expression. Resolution matched the name alone, so
every handler protoc-gen-go-grpc emits went to the `UnimplementedKVServer`
stub beside the `KVServer` interface (72 edges in etcd), and calls like
`c.Reader.(*pipe).Close()`, `p.(Pausable).Pause()` or
`v.(featuregate.MutableFeatureGate).Set(…)` linked to a namesake or to
nothing.

The call's chain is now read back from its column (operand, selectors,
assertions, calls and indexes, past strings, runes and comments, and on to
the next line after a trailing `.`). When the link of the call's name
follows an assertion, the call is a method of the asserted type, found
where Go finds it:
- a bare `T` or `*T` in the call's own package, or one it dot-imports;
- a `pkg.T` in the imported project package;
- its own method, the one its interface declares, or one promoted from a
  type it embeds, through the package-scoped lookup #2361 added; an alias
  is followed to the type it names (#2417).

A type from outside the project (`http.Flusher`), a predeclared one
(`error`), an alias of an outside type (`type Ctx = context.Context`, for
which the package-scoped lookup would fall back to matching by name) or a
type literal (`interface{ Flush() }`) links nothing. The check runs ahead
of the framework, import and name strategies, and ahead of the built-in
filter, which dropped a method named `close` or `copy` called through an
assertion. Two calls of one name in a chain share the column
(`b.(*Builder).Add(1).Add(2)`); both are taken for the one made through the
assertion, whose edge is there either way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant