Skip to content
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
- In C++, a class written with attribute macros the parser can't read now keeps its members. That covers several macros between `class` and the class name, like protocolbuffers/protobuf's generated `class PROTOBUF_EXPORT PROTOBUF_FUTURE_ADD_EARLY_WARN_UNUSED Any final`, a macro with arguments there, like RocksDB's `struct ALIGN_AS(64U) HandleImpl`, or one before a template specialization, and macros on a member: in front of it (`PROTOBUF_FUTURE_ADD_EARLY_NODISCARD`), between a pointer and its name (`PROTOBUF_NONNULL`), after its parameters or its name (`ABSL_ATTRIBUTE_LIFETIME_BOUND`, google/leveldb's `LOCKS_EXCLUDED(mu_)` and `GUARDED_BY(mu_)`), or alone on a line above a comment ({fmt}'s `FMT_BEGIN_EXPORT`). Before, protobuf's generated message classes were missing and their members, like `GetDescriptor`, `GetReflection` and `unknown_fields`, were indexed as functions outside any class, and leveldb's annotated methods were indexed under the annotation's name, so calls to them linked to nothing. Re-index C++ projects after upgrading.
- Framework support now switches on when an app's `package.json` sits three or more folders deep, as with an ASP.NET solution's Angular or React app in `src/Web/ClientApp/`, or a React app in a workspace folder like `web/ui/mantine-ui/`. Before, only the project root and the first two folder levels were read, so frameworks such as Angular, React Router, Vue Router, SvelteKit and TanStack Router could go undetected there: an Angular app's routes, its `routerLink` and `router.navigate(…)` navigation and its template's event bindings were missing, so a handler used only from a `(click)` binding had no callers, and a React app's `navigate('/login')` calls didn't reach their route. Re-index projects laid out this way after upgrading.
- In a repository that mixes languages, a web framework's naming conventions now link only code written in that framework's own languages. Before, Express's also linked Go and Python code: in etcd a Go method's `*zap.Logger` result type linked to the method itself, and in Harbor a Python `self.jaeger.validate()` call linked to another class's `validate`. Links that cross languages on purpose, like React Native and Expo native modules, Swift and Objective-C, Drupal's `routing.yml`, SvelteKit's `$lib/…` imports in `.ts` files and a Razor page's `@model`, are unchanged.
- In C and C++, a call written from the global scope, like `::leveldb::RepairDB(…)` or `::hpb::CreateMessage(…)`, now links to the function of exactly that name, and one through a namespace alias, like protobuf's `::_pbi::…` calls, to the function in the aliased namespace. Before, these calls were dropped unless the repository also held Swift and Objective-C files, and even then a name spelled out in full, like `::google::protobuf::internal::TSanWrite(…)`, linked to nothing, while `::operator delete(…)` linked to one class's own `operator delete`. A global call never links to a function or method of the same name inside a namespace or class, such as a wrapper `open()` that calls the C library's `::open(…)`, and `::std::…` and `::testing::…` calls stay unlinked unless that library's source is part of your project. Re-index C and C++ projects after upgrading.
- In Go, an interface that embeds another, like gin's `IRouter` embedding `IRoutes`, now links to it, so `codegraph_explore` and impact show the embedded interface as its base. Before, every embedded interface was dropped. An embedded field written through another package of your project, like `*refresh.Discovery` or `storage.Appender`, or with type arguments, like `List[int]`, now links too. One from the standard library or another module, like `io.Closer` or `sync.Mutex`, links to nothing rather than to a project type that shares its name, and a type-set constraint like `~int | ~float64` is not an embedding. An embedded type written without a package now always links to its own package's type: before, etcd's `Client`, which embeds its own `Lease` interface, was linked to a `Lease` struct in another package. Re-index Go projects after upgrading.
- In Go, working out which structs implement an interface now counts the methods of the interfaces it embeds, and the methods a struct gets from the types it embeds. Before, only each type's own methods counted: etcd's `AuthReadTx`, which embeds `UnsafeAuthReader` and adds `RLock` and `RUnlock`, was listed as implemented by every read-write lock in the project, an interface that only embeds others had no implementations at all, and a struct that gets its methods from an embedded base, like prometheus's service discoveries embedding `refresh.Discovery` or gin's `Engine` embedding `RouterGroup`, was missing. `codegraph_explore`, impact and the type hierarchy now list the right implementations, and a call through the interface reaches a method the embedding struct overrides, like gin's `Engine.Use`. Re-index Go projects after upgrading.
- In Go, a call through an interface now reaches a method that the implementing struct gets from a type it embeds: prometheus's `targetScraper` gets `Report` from the `*Target` it embeds, so a call through the `scraper` interface now reaches `Target.Report`, and etcd's `Hasher.MemberID` reaches `EtcdServer.MemberID` through `hasherAdapter`. Before, the call reached only structs that declare the method themselves, often just a test double, so `codegraph_explore`'s call path stopped at the interface. The method is picked the way Go picks it, from the nearest embedded type, and none is linked when two embedded types at the same depth both have it or it comes from an embedded interface. Re-index Go projects after upgrading.
Expand Down
197 changes: 197 additions & 0 deletions __tests__/cpp-global-scope-names.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,197 @@
/**
* A C or C++ name written from the global scope — `::store::Repair(…)`,
* protobuf's `::_pbi::PrivateAccess::GenerateParseTable(…)` through
* `namespace _pbi = ::google::protobuf::internal;` — never got past the
* resolver's name-existence pre-filter, which read no leading `::`. protobuf's
* calls resolved only because it ships a few Swift and Objective-C files, and
* the Swift ↔ Objective-C bridge claimed every name with a `:` in it; C++-only
* rocksdb, leveldb and fmt dropped theirs. Past the pre-filter, the name is the
* declaration of exactly that qualified name: never one nested in another
* namespace or class, as the wrapper `int open(…) { return ::open(…); }` and an
* external `::testing::` call show.
*/
import { describe, it, expect, afterAll, beforeAll } from 'vitest';
import * as fs from 'fs';
import * as os from 'os';
import * as path from 'path';
import { CodeGraph } from '../src';

function writeProject(prefix: string, files: Record<string, string>): string {
const root = fs.mkdtempSync(path.join(os.tmpdir(), prefix));
for (const [rel, content] of Object.entries(files)) {
fs.mkdirSync(path.dirname(path.join(root, rel)), { recursive: true });
fs.writeFileSync(path.join(root, rel), content);
}
return root;
}

/**
* `name -> target qualified name` for every reference the named function or
* method in `file` resolves — a call to a struct is an `instantiates` edge.
*/
function callsFrom(cg: CodeGraph, file: string, name: string): string[] {
const from = cg.getNodesInFile(file).filter((n) => n.name === name && (n.kind === 'function' || n.kind === 'method'));
expect(from.length).toBeGreaterThan(0);
return cg
.getOutgoingEdgesFrom(from.map((n) => n.id))
.flatMap((e) => {
const refName = (e.metadata as { refName?: string } | undefined)?.refName;
return refName ? [`${refName} -> ${cg.getNode(e.target)!.qualifiedName}`] : [];
})
.sort();
}

const PB_INTERNAL = `#pragma once
namespace pb {
namespace internal {
inline int Prefetch(const void* ptr) { return ptr != nullptr; }
} // namespace internal
} // namespace pb

namespace _pbi = ::pb::internal;
`;

describe('C++ names written from the global scope', () => {
let root = '';
let cg: CodeGraph;

beforeAll(async () => {
root = writeProject('cg-cpp-global-', {
'src/store/repair.cc': `namespace store {
int Repair(const char* name) { return name != nullptr; }
int move(int value) { return value; }

namespace testing {
const char* TempDir() { return "/tmp/store"; }
} // namespace testing
} // namespace store
`,
'src/pb/internal.h': PB_INTERNAL,
'src/store/posix.cc': `#include <unistd.h>

namespace store {
namespace posix {
int close(int fd) { return ::close(fd); }
} // namespace posix
} // namespace store
`,
'tools/repair_tool.cc': `#include <utility>
#include "pb/internal.h"

namespace tools {
namespace store {
int Repair(const char* name) { return 0; }
} // namespace store

int Run(const char* name) {
int fixed = ::store::Repair(name);
int warmed = ::_pbi::Prefetch(name);
return ::std::move(fixed) + warmed;
}
} // namespace tools
`,
'include/lib/os.h': `#pragma once
#define LIB_BEGIN_NAMESPACE \\
namespace lib { \\
inline namespace v1 {
#define LIB_END_NAMESPACE \\
} \\
}

LIB_BEGIN_NAMESPACE
struct pipe {
int read_end;
int write_end;
};
LIB_END_NAMESPACE
`,
'tests/posix-mock.cc': `#include <unistd.h>
#include "lib/os.h"

namespace test {
int pipe(int fds[2]) { return ::pipe(fds); }
} // namespace test
`,
'tests/repair_test.cc': `#include <gtest/gtest.h>

class RepairTest : public ::testing::Test {};

int RunAll(int argc, char** argv) {
::testing::InitGoogleTest(&argc, argv);
return ::testing::TempDir() != nullptr;
}
`,
});
cg = await CodeGraph.init(root, { index: true });
}, 60_000);

afterAll(() => {
cg?.close();
if (root) fs.rmSync(root, { recursive: true, force: true });
});

it('links `::ns::f()` to the global namespace’s f, and an alias’s call to the aliased namespace', () => {
// Inside `namespace tools`, `store::Repair` would be tools' own; `::store::Repair` is the global one.
expect(callsFrom(cg, 'tools/repair_tool.cc', 'Run')).toEqual([
'::_pbi::Prefetch -> pb::internal::Prefetch',
'::store::Repair -> store::Repair',
]);
});

it('never links a global name to a namesake nested in a namespace or class', () => {
// The wrapper calls the C library's `close`, not itself.
expect(callsFrom(cg, 'src/store/posix.cc', 'close')).toEqual([]);
// Nor `lib::pipe`, whose namespace a macro opens (fmt's FMT_BEGIN_NAMESPACE):
// the index cannot see it, so the struct only looks global there.
expect(callsFrom(cg, 'tests/posix-mock.cc', 'pipe')).toEqual([]);
// gtest is not in the project: `::testing::TempDir` is not `store::testing::TempDir`.
expect(callsFrom(cg, 'tests/repair_test.cc', 'RunAll')).toEqual([]);
});

it('leaves an external `::testing::Test` base unresolved', () => {
const test = cg.getNodesInFile('tests/repair_test.cc').find((n) => n.name === 'RepairTest')!;
expect(test).toBeDefined();
expect(cg.getOutgoingEdges(test.id).filter((e) => e.kind === 'extends' || e.kind === 'implements')).toEqual([]);
});
});

describe('C++ global names beside Swift and Objective-C (protobuf)', () => {
let root = '';
let cg: CodeGraph;

beforeAll(async () => {
root = writeProject('cg-cpp-global-objc-', {
'objectivec/GPBLegacy.m': `#import <Foundation/Foundation.h>

@interface GPBLegacy : NSObject
- (void)start;
@end

@implementation GPBLegacy
- (void)start {}
@end
`,
'swift/Bridge.swift': `import Foundation

final class Bridge: NSObject {
func run() {}
}
`,
'src/pb/internal.h': PB_INTERNAL,
'src/message.cc': `#include "pb/internal.h"

int Parse(const void* data) { return ::_pbi::Prefetch(data); }
`,
});
cg = await CodeGraph.init(root, { index: true });
}, 60_000);

afterAll(() => {
cg?.close();
if (root) fs.rmSync(root, { recursive: true, force: true });
});

it('still links a C++ `::_pbi::` call, which no longer rides the Swift ↔ Objective-C bridge’s claim', () => {
expect(callsFrom(cg, 'src/message.cc', 'Parse')).toEqual(['::_pbi::Prefetch -> pb::internal::Prefetch']);
});
});
7 changes: 5 additions & 2 deletions docs/design/framework-coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -313,8 +313,11 @@ Each of these cost real debugging time; they are not hypothetical.
A resolver that reads a language it extracts nothing from lists it in
`resolveLanguages` instead — Svelte's `$lib/…` imports in `.ts` route
modules, ASP.NET's Razor `@model` — because widening `languages` also runs
`extract()` on those files. `claimsReference()` is still asked of every
detected framework.
`extract()` on those files. `claimsReference()` is asked of the same
frameworks. It used to be asked of every detected one, because protobuf's
C++ `::_pbi::…` calls got past the name pre-filter only on the Swift ↔
Objective-C bridge's claim of any name with a `:` in it. The pre-filter
now reads a leading `::` as the global scope.
16. **A navigation call waits for a route, not for a name.** A call indexed
before its route existed is parked as failed (or bound to a catch-all, a
parameter route, the other arm of a conditional), and sync revisits a
Expand Down
20 changes: 12 additions & 8 deletions src/resolution/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1301,6 +1301,11 @@ export class ReferenceResolver {
// Erlang refs carry the call-site arity (`f/1`, `mod::f/2` — #1610); the
// name index stores bare names, so existence is checked arity-less.
if (ref.language === 'erlang') existenceName = existenceName.replace(/\/\d{1,3}$/, '');
// A C or C++ name written from the global scope (`::_pbi::PrivateAccess::
// GenerateParseTable`, `::memset`) exists when the name under the `::` does.
if ((ref.language === 'cpp' || ref.language === 'c') && existenceName.startsWith('::')) {
existenceName = existenceName.slice(2);
}
const tPre = this.profileStages ? process.hrtime.bigint() : 0n;
const preFilterPass =
isNixPathImportRef(ref) ||
Expand All @@ -1310,11 +1315,7 @@ export class ReferenceResolver {
// calls `FormatPrice`, which the exact-name set never lists.
(CASE_INSENSITIVE_LANGUAGES.has(ref.language) && this.hasAnyPossibleMatchIgnoringCase(existenceName)) ||
this.matchesAnyImport(ref) ||
// Every detected framework's claim, not only those that resolve this
// language: the check above reads no leading `::`, and protobuf's C++
// `::_pbi::…` calls get past it only on the Swift ↔ Objective-C
// bridge's claim of any name with a `:` in it.
this.frameworks.some((f) => f.claimsReference?.(ref.referenceName));
this.frameworksFor(ref.language).some((f) => f.claimsReference?.(ref.referenceName));
if (this.profileStages) this.stageAdd('preFilter', ref, preFilterPass, tPre);
if (!preFilterPass) {
return this.gateLanguage(matchJsStoreBindingCall(ref, this.context), ref);
Expand Down Expand Up @@ -2833,12 +2834,15 @@ export class ReferenceResolver {
// when there's no user node with this name — then name-matching would
// produce zero edges anyway and the filter just short-circuits work.
if (ref.language === 'c' || ref.language === 'cpp') {
// A leading `::` names the global scope: `::std::move` is `std::move`,
// `::memset` is `memset`.
const scoped = name.startsWith('::') ? name.slice(2) : name;
// C++ std:: namespace prefix — safe to filter unconditionally,
// since `std::foo` is never a user-defined qualified name in
// tree-sitter output.
if (name.startsWith('std::')) return true;
if (C_BUILT_INS.has(name) || CPP_BUILT_INS.has(name)) {
return !this.hasAnyPossibleMatch(name);
if (scoped.startsWith('std::')) return true;
if (C_BUILT_INS.has(scoped) || CPP_BUILT_INS.has(scoped)) {
return !this.hasAnyPossibleMatch(scoped);
}
}

Expand Down
22 changes: 22 additions & 0 deletions src/resolution/name-matcher.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7811,6 +7811,26 @@ export function matchByQualifiedName(
)
: nodes;

// A C or C++ name written from the global scope (`::hpb::CreateMessage`,
// `::operator delete`) is the declaration of exactly that name. The suffix
// match below would take one nested in another namespace or class: for
// fmt's mock `test::open`, the `::open(…)` it wraps. Nor is it a declaration
// inside a namespace a macro opens, which only looks global in the index
// (fmt's `struct pipe`); step 5 of matchReference reads those.
if ((ref.language === 'cpp' || ref.language === 'c') && ref.referenceName.startsWith('::')) {
const global = keepForRef(context.getNodesByQualifiedName(ref.referenceName.slice(2))).filter((n) =>
(n.language === 'cpp' || n.language === 'c') &&
!cppMacroNamespaceFrames(n.filePath, context).some((f) => f.start <= n.startLine && f.end >= n.startLine));
const chosen = preferCallSiteFile(global, ref.filePath)[0];
if (!chosen) return null;
return {
original: ref,
targetNodeId: chosen.id,
confidence: global.length === 1 || chosen.filePath === ref.filePath ? 0.95 : 0.85,
resolvedBy: 'qualified-name',
};
}

let candidates = keepForRef(context.getNodesByQualifiedName(ref.referenceName));
// A C# `using X.Y;` names a namespace: one the project declares, else it is
// the file's own (external) using — never another file's using of that name.
Expand Down Expand Up @@ -13732,6 +13752,8 @@ export function cppMacroNamespaceFrames(file: string, context: ResolutionContext
*/
function matchCppMacroNamespaced(ref: UnresolvedRef, context: ResolutionContext): ResolvedRef | null {
let target = ref.referenceName.replace(/^::/, '');
// `::memset` names no namespace for a macro or an alias to open.
if (!target.includes('::')) return null;
const head = target.slice(0, target.indexOf('::'));
const alias = cppNamespaceAliases(context).get(head);
// `py::str` under `namespace py = pybind11;` is `pybind11::str`.
Expand Down
1 change: 1 addition & 0 deletions src/resolution/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -319,6 +319,7 @@ export interface FrameworkResolver {
* an attribute/descriptor, not a declared symbol (e.g. Django's
* `self._iterable_class(...)`, React effect callbacks). Returning true lets the
* ref reach `resolve()` instead of being dropped for having no name match.
* Asked only about references written in the languages `resolve()` sees.
*/
claimsReference?(name: string): boolean;
/**
Expand Down