diff --git a/CHANGELOG.md b/CHANGELOG.md index dd5a43e916..d67127da8c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -470,6 +470,9 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). - In C++, a call on the variable of a range-based `for` loop over `auto` elements, like `cfd->Ref()` inside facebook/rocksdb's `for (auto cfd : cfds)`, now links to the method of the type the loop goes over. That type is read from what the loop iterates: a `std::vector`, `std::set`, `absl::Span` or similar container, protocolbuffers/protobuf's `RepeatedPtrField` or RocksDB's `autovector`, whether a variable, parameter or member holds it, an `auto` variable is set to it or a function or method returns it, and a project class you loop over through its `begin()`, like RocksDB's `ColumnFamilySet`. Before, the call was matched by the variable's name, or by an earlier variable of that name: in RocksDB, `blob_file.GetBlobFileNumber()` over a list of `BlobFileGarbage` reached `BlobFileAddition::GetBlobFileNumber`, and in protobuf's Objective-C generator, calls over a list of `std::unique_ptr` reached other generators' methods. A call on an element of a standard-library type, like `file.find(…)` over a `std::vector`, no longer links to a project method, and `->` on an element held in a `std::unique_ptr` or `std::shared_ptr` reaches the type it holds. Re-index C++ projects after upgrading. - In C++, a call on a variable, parameter or member declared as one of your classes, like `iter->Valid()` on an `Iterator* iter` or `it.Valid()` on a `const Iterator& it`, now links to that method of the class C++ itself would find for the type where the call is written, or of a class it inherits from. Before, the method was picked by the type's last name, so a class of the same name nested elsewhere could win: in facebook/rocksdb, calls on an `Iterator` reached `MemTableRep::Iterator`, which declares `Valid` and `Next` itself while the `Iterator` the code means inherits them, google/leveldb's reached `SkipList::Iterator`, and in protocolbuffers/protobuf the Java generator's `context->options()` reached the hpb generator's `Context` and `text_parser.Parse(…)` on a `TextFormat::Parser` reached the `.proto` parser. When that class and the classes it derives from have no method of that name, the call no longer links to an unrelated class's method that shares it, as protobuf's `field->full_name()` on a `FieldDescriptor*` did. A type is also found through a `using` declaration the file writes, like protobuf's `using google::protobuf::Descriptor;`, before a class of the same name declared at global scope elsewhere in the repository; this also applies to base classes and to the type a smart pointer holds. Re-index C++ projects after upgrading. - In this fork, the two C++ changes above run in the native resolver. A call on a variable whose type is a template parameter, like `T* iter`, stays unlinked rather than reaching a class that happens to share the parameter's name. A call stays linked as before when the class has no method of that name but one of its base classes comes from an outside library other than the standard library, such as googletest's `testing::Test`. +- In C++, a call on an `auto` variable now links to the method of the class C++ finds for the type its initializer gives it, as a call on a variable declared with that type does. The type is read from a cast or smart pointer written out, like facebook/rocksdb's `auto iter = std::unique_ptr(…)` or `auto* mutex = static_cast(arg)`, from `new` or `std::make_unique` of a class, and from what a function or method is declared to return, like the `Iterator*` of `txn->GetIterator(…)`, read to the end of an initializer that goes on to the next line. Before, such a variable took the type of an earlier variable of the same name, or none: in RocksDB, `iter->SeekToFirst()` reached `MemTableRep::Iterator` and `mutex->Lock()` reached `port::Mutex`. A chained call like `cfd->current()->storage_info()` now looks up the class the inner call is declared to return the same way, through a smart pointer too, so protocolbuffers/protobuf's Java generator no longer reaches the C# generator's `EnumGenerator`. A type from outside the project no longer links to a class of the project that shares its name: protobuf's `status.ok()` on an `absl::Status` reached `upb::Status::ok`, and fmtlib/fmt's `os.flush()` on a `std::ostream&` reached fmt's own `ostream::flush`. Re-index C++ projects after upgrading. +- In C++, a call on a data member of the calling method's class, like `versions_->GetColumnFamilySet()` in facebook/rocksdb's `db_impl_write.cc`, now links to the method of the member's type wherever the class is declared. Before, only the call's own file and the header of the same name were read for the member, and RocksDB declares `DBImpl` in `db_impl.h` while its methods are spread over `db_impl_write.cc`, `db_impl_compaction_flush.cc`, `db_impl_open.cc` and more, so such calls were guessed by the member's name. A member declared below an inline method that uses it, one a base class declares, and, in a gtest `TEST_F` or `TEST_P` body, one of the test's fixture, like RocksDB's `db_`, are found the same way, and a member of a class nested in the caller's class is no longer taken for the caller's own. A call on a member of a standard-library type, like `std::atomic shutting_down_`, no longer links to a project method that only shares the method's name. Re-index C++ projects after upgrading. +- In this fork, the two C++ changes above also run in the native resolver. A call on a variable, member or chain whose type it cannot read, such as one a template parameter decides (`T*`, `std::shared_ptr`), an alias of a trait of a parameter, or the result of a function template whose return type is its parameter, stays unlinked rather than being guessed by the receiver's name, and a chain of up to four calls (`cfd->current()->storage_info()->LevelFiles()`) is read from its declarations, not only its first inner call; a longer one is looked up by the last name, as before. A chained call spelled with `.` on a returned `std::unique_ptr` or `std::optional` is that type's own member, not a method of what it holds (`->` still reaches that); and a variable declared in the initializer of a `for`, `if` or `switch` statement is seen before a same-named member the class inherits. - Framework support now switches on when an app's `package.json` sits three or more folders deep, as with an ASP.NET solution's Angular or React app in `src/Web/ClientApp/`, or a React app in a workspace folder like `web/ui/mantine-ui/`. Before, only the project root and the first two folder levels were read, so frameworks such as Angular, React Router, Vue Router, SvelteKit and TanStack Router could go undetected there: an Angular app's routes, its `routerLink` and `router.navigate(…)` navigation and its template's event bindings were missing, so a handler used only from a `(click)` binding had no callers, and a React app's `navigate('/login')` calls didn't reach their route. Re-index projects laid out this way after upgrading. - In Go, an interface that embeds another, like gin's `IRouter` embedding `IRoutes`, now links to it, so `codegraph_explore` and impact show the embedded interface as its base. Before, every embedded interface was dropped. An embedded field written through another package of your project, like `*refresh.Discovery` or `storage.Appender`, or with type arguments, like `List[int]`, now links too. One from the standard library or another module, like `io.Closer` or `sync.Mutex`, links to nothing rather than to a project type that shares its name, and a type-set constraint like `~int | ~float64` is not an embedding. An embedded type written without a package now always links to its own package's type: before, etcd's `Client`, which embeds its own `Lease` interface, was linked to a `Lease` struct in another package. Re-index Go projects after upgrading. diff --git a/README.md b/README.md index 0db7f309b7..3c22ed4ed5 100644 --- a/README.md +++ b/README.md @@ -52,7 +52,7 @@ Follow [@getcodegraph](https://x.com/getcodegraph) on X for updates. ## About this fork -This is **bompus/codegraph**, a fork of [colbymchenry/codegraph](https://github.com/colbymchenry/codegraph). Its default branch, `fork/consolidated`, contains upstream `main` through [`c19f0284`](https://github.com/colbymchenry/codegraph/commit/c19f0284) (after v1.6.2) plus the fork's own work, and it takes upstream changes as they land. Changes that suit upstream are also offered there as pull requests. +This is **bompus/codegraph**, a fork of [colbymchenry/codegraph](https://github.com/colbymchenry/codegraph). Its default branch, `fork/consolidated`, contains upstream `main` through [`df3ce972`](https://github.com/colbymchenry/codegraph/commit/df3ce972) (after v1.6.2) plus the fork's own work, and it takes upstream changes as they land. Changes that suit upstream are also offered there as pull requests. The fork publishes no releases. The install scripts, npm package, badges and `codegraph upgrade` further down this page install **upstream's** releases. To run the fork, build it from source (below). @@ -152,6 +152,7 @@ Call resolution the fork adds: - Kotlin multi-dollar strings keep their interpolation threshold. - C++ typedef and using aliases follow their declaration scope, including nested owners and inherited aliases. Pointer, array and function members are not supertypes. Visible class-scoped aliases and declared complex receivers keep their method owners; unsubstituted template parameters and ambiguous owners stay unresolved. - C++ `->` calls on a `std::unique_ptr`, `std::shared_ptr` or `std::optional` reach the type it holds, looked up through the caller's enclosing classes and their bases, its namespaces, `using` directives and aliases. A held type the project does not declare leaves the call unresolved, as a plain pointer to one does; upstream guesses a callee from the receiver's name there. A partial class template specialization is indexed as an anonymous class, so a call to a member only it declares stays unresolved. +- C++ calls on an `auto` local, on a chained call and on a data member of the caller's class (one a base class declares, or a gtest `TEST_F` / `TEST_P` fixture's) take the type its declaration writes, as upstream's do. Where that type cannot be read (a template parameter's, a `std::shared_ptr` of one, a trait of one, what a function template returns), the call stays unlinked, as does a local of a type the file does not declare; upstream guesses a callee from the receiver's name at 0.7. A chain of up to four calls is read from its declarations, where upstream reads its first inner call; a longer one is looked up by the last name, as before. - C++ namespace aliases and declarations in macro-opened namespaces are looked up only in the caller's include closure; upstream pools them across all files. - C# namespace `using` directives and `using` aliases apply only inside their enclosing namespace, not to sibling namespaces in the same file (`using static` is still file-wide, as upstream). - A Java field declared with a qualified type (`outside.Repository`) keeps its qualifier, so calls through it never resolve to an unrelated project class with the same simple name. diff --git a/__tests__/cpp-auto-local-receiver.test.ts b/__tests__/cpp-auto-local-receiver.test.ts new file mode 100644 index 0000000000..19ab514fe9 --- /dev/null +++ b/__tests__/cpp-auto-local-receiver.test.ts @@ -0,0 +1,774 @@ +/** + * A C++ call on a receiver whose type C++ lookup finds reaches that class, + * not a namesake; and a receiver of a type from outside the project reaches + * none of the project's classes. + * + * #2493 resolved calls on declared pointers, values and references by C++ + * lookup. An `auto` local was still typed by the last name of its + * initializer's type, when that could be read at all: in facebook/rocksdb, + * `auto iter = std::unique_ptr(…)` couldn't be, so the scan read an + * earlier function's `Iterator* iter`, and its last name took + * `iter->SeekToFirst()` to the nested `MemTableRep::Iterator`. The local now + * has the type its initializer has as written where it is declared: a + * construction or cast names it; a call's declaration writes it. Its calls + * then resolve as for a declared receiver. + * + * A type from outside the project (`absl::Status`, `std::ostream`) took the + * project's class of the same last name: protocolbuffers/protobuf's + * `upb::Status`, fmtlib/fmt's own `ostream`. + * + * A guess can still land on the right method by a shared word, so these + * tests check the confidence, not only the target. + */ +import { describe, it, expect, afterEach } from 'vitest'; +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import { CodeGraph } from '../src'; +import type { Node } from '../src/types'; + +const roots: string[] = []; +afterEach(() => { + for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true }); +}); + +async function indexed(files: Record): Promise { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cpp-auto-local-')); + roots.push(root); + for (const [rel, content] of Object.entries(files)) { + fs.mkdirSync(path.dirname(path.join(root, rel)), { recursive: true }); + fs.writeFileSync(path.join(root, rel), content); + } + return CodeGraph.init(root, { index: true }); +} + +function callable(cg: CodeGraph, qualifiedName: string): Node { + const node = [...cg.getNodesByKind('method'), ...cg.getNodesByKind('function')].find((n) => n.qualifiedName === qualifiedName); + if (!node) throw new Error(`no function or method ${qualifiedName}`); + return node; +} + +/** `calls` callees of a function or method, as `qualifiedName @confidence`, one per call site. */ +function calls(cg: CodeGraph, caller: string): string[] { + return cg + .getCallees(callable(cg, caller).id) + .filter((r) => r.edge.kind === 'calls') + .map((r) => `${r.node.qualifiedName} @${(r.edge.metadata as { confidence?: number } | undefined)?.confidence}`) + .sort(); +} + +const lines = (...text: string[]): string => [...text, ''].join('\n'); + +/** rocksdb's iterators, trimmed: the public Iterator inherits its cursor methods; a nested namesake declares them. */ +const ITERATORS = { + 'include/rocksdb/iterator_base.h': lines( + 'namespace rocksdb {', + 'class IteratorBase {', + ' public:', + ' virtual ~IteratorBase() {}', + ' virtual bool Valid() const = 0;', + ' virtual void SeekToFirst() = 0;', + ' virtual void Next() = 0;', + '};', + '} // namespace rocksdb', + ), + 'include/rocksdb/iterator.h': lines( + '#include "include/rocksdb/iterator_base.h"', + 'namespace rocksdb {', + 'class Iterator : public IteratorBase {', + ' public:', + ' virtual int value() const = 0;', + '};', + '} // namespace rocksdb', + ), + 'include/rocksdb/memtablerep.h': lines( + 'namespace rocksdb {', + 'class MemTableRep {', + ' public:', + ' class Iterator {', + ' public:', + ' virtual bool Valid() const = 0;', + ' virtual void SeekToFirst() = 0;', + ' virtual void Next() = 0;', + ' };', + '};', + '} // namespace rocksdb', + ), + 'include/rocksdb/db.h': lines( + '#include "include/rocksdb/iterator.h"', + 'namespace rocksdb {', + 'class DB {', + ' public:', + ' virtual Iterator* NewIterator() = 0;', + '};', + 'class Transaction {', + ' public:', + ' virtual Iterator* GetIterator() = 0;', + '};', + '} // namespace rocksdb', + ), +}; + +describe('an `auto` local has the type its initializer has where that is written', () => { + it('reproduction: a smart pointer cast to, and what a method is declared to return', async () => { + const cg = await indexed({ + ...ITERATORS, + 'db/db_iterator_test.cc': lines( + '#include ', + '#include "include/rocksdb/db.h"', + '#include "include/rocksdb/memtablerep.h"', + 'namespace rocksdb {', + 'int CountKeys(DB* db) {', + ' int n = 0;', + ' auto iter = std::unique_ptr(db->NewIterator());', + ' for (iter->SeekToFirst(); iter->Valid(); iter->Next()) n += iter->value();', + ' return n;', + '}', + 'bool FirstValid(Transaction* txn) {', + ' auto it = txn->GetIterator();', + ' it->SeekToFirst();', + ' return it->Valid();', + '}', + '} // namespace rocksdb', + ), + }); + try { + // IteratorBase's, which Iterator derives from; not MemTableRep::Iterator's. + expect(calls(cg, 'rocksdb::CountKeys')).toEqual([ + 'rocksdb::DB::NewIterator @0.9', + 'rocksdb::Iterator::value @0.9', + 'rocksdb::IteratorBase::Next @0.9', + 'rocksdb::IteratorBase::SeekToFirst @0.9', + 'rocksdb::IteratorBase::Valid @0.9', + ]); + expect(calls(cg, 'rocksdb::FirstValid')).toEqual([ + 'rocksdb::IteratorBase::SeekToFirst @0.9', + 'rocksdb::IteratorBase::Valid @0.9', + 'rocksdb::Transaction::GetIterator @0.9', + ]); + } finally { + cg.close(); + } + }); + + it("a smart pointer to a class template of the project's, whatever its arguments", async () => { + // rocksdb's ExternalTableReaderAdapter holds an ExternalTableReaderBase. + const cg = await indexed({ + 'table/external_table.cc': lines( + '#include ', + 'namespace rocksdb {', + 'template ', + 'class ExternalTableReaderBase {', + ' public:', + ' virtual ~ExternalTableReaderBase() {}', + ' int NewIterator() { return Mode; }', + '};', + 'class IndexFactoryReader {', + ' public:', + ' int NewIterator() { return 0; }', + '};', + 'template ', + 'class ExternalTableReaderAdapter {', + ' public:', + ' int Open() {', + ' auto reader = std::unique_ptr>(nullptr);', + ' return reader->NewIterator();', + ' }', + '};', + '} // namespace rocksdb', + ), + }); + try { + expect(calls(cg, 'rocksdb::ExternalTableReaderAdapter::Open')).toEqual(['rocksdb::ExternalTableReaderBase::NewIterator @0.9']); + } finally { + cg.close(); + } + }); + + it('a cast, and a construction through a typedef', async () => { + // rocksdb's sync-point callbacks cast their argument; HashSkipListRep's + // `Bucket` is a typedef of a SkipList. + const cg = await indexed({ + 'monitoring/instrumented_mutex.h': lines( + 'namespace rocksdb {', + 'class InstrumentedMutex {', + ' public:', + ' void Lock() {}', + ' void Unlock() {}', + '};', + 'namespace port {', + 'class Mutex {', + ' public:', + ' void Lock() {}', + ' void Unlock() {}', + '};', + '} // namespace port', + '} // namespace rocksdb', + ), + 'memtable/skiplist.h': lines( + 'namespace rocksdb {', + 'template ', + 'class SkipList {', + ' public:', + ' void Insert(const Key& key) {}', + '};', + 'class InlineSkipList {', + ' public:', + ' void Insert(const char* key) {}', + '};', + '} // namespace rocksdb', + ), + 'db/merge_test.cc': lines( + '#include "monitoring/instrumented_mutex.h"', + '#include "memtable/skiplist.h"', + 'namespace rocksdb {', + 'void Callback(void* arg) {', + ' auto* mutex = static_cast(arg);', + ' mutex->Unlock();', + ' mutex->Lock();', + '}', + 'class HashSkipListRep {', + ' public:', + ' typedef SkipList Bucket;', + ' void Add(const char* key) {', + ' auto list = new Bucket();', + ' list->Insert(key);', + ' }', + '};', + '} // namespace rocksdb', + ), + }); + try { + expect(calls(cg, 'rocksdb::Callback')).toEqual(['rocksdb::InstrumentedMutex::Lock @0.9', 'rocksdb::InstrumentedMutex::Unlock @0.9']); + expect(calls(cg, 'rocksdb::HashSkipListRep::Add')).toEqual(['rocksdb::SkipList::Insert @0.9']); + } finally { + cg.close(); + } + }); + + it("a bare call to a method the caller's class inherits, and a smart pointer made for a class", async () => { + // protobuf's `auto* h = micro_rep();` read an earlier function's + // `LargeRep* h`; a test helper's `Widget` shares a name with the UI's. + const cg = await indexed({ + 'src/google/protobuf/micro_string.h': lines( + '#include ', + 'namespace google {', + 'namespace protobuf {', + 'struct LargeRep {', + ' void ChangeSize(int n) {}', + '};', + 'struct MicroRep {', + ' void ChangeSize(int n) {}', + '};', + 'class MicroStringBase {', + ' protected:', + ' MicroRep* micro_rep() { return nullptr; }', + ' LargeRep* large_rep() { return nullptr; }', + '};', + 'class MicroString : public MicroStringBase {', + ' public:', + ' void Grow();', + ' void Clear();', + '};', + 'namespace ui {', + 'class Widget {', + ' public:', + ' void Draw() {}', + '};', + '} // namespace ui', + '} // namespace protobuf', + '} // namespace google', + ), + 'src/google/protobuf/micro_string.cc': lines( + '#include "src/google/protobuf/micro_string.h"', + 'namespace google {', + 'namespace protobuf {', + 'void MicroString::Grow() {', + ' LargeRep* h = large_rep();', + ' h->ChangeSize(2);', + '}', + 'namespace test {', + 'class Widget {', + ' public:', + ' void Draw() {}', + '};', + '} // namespace test', + 'void MicroString::Clear() {', + ' auto* h = micro_rep();', + ' h->ChangeSize(0);', + '}', + 'namespace ui {', + 'void Paint() {', + ' auto w = std::make_unique();', + ' w->Draw();', + '}', + 'void Repaint() {', + ' auto w = Widget();', + ' w.Draw();', + '}', + '} // namespace ui', + '} // namespace protobuf', + '} // namespace google', + ), + }); + try { + expect(calls(cg, 'google::protobuf::MicroString::Clear')).toEqual([ + 'google::protobuf::MicroRep::ChangeSize @0.9', + 'google::protobuf::MicroStringBase::micro_rep @0.9', + ]); + expect(calls(cg, 'google::protobuf::ui::Paint').filter((c) => c.includes('Draw'))).toEqual(['google::protobuf::ui::Widget::Draw @0.9']); + expect(calls(cg, 'google::protobuf::ui::Repaint').filter((c) => c.includes('Draw'))).toEqual(['google::protobuf::ui::Widget::Draw @0.9']); + } finally { + cg.close(); + } + }); + + it('a chained call through a smart pointer reaches a method of the class it holds', async () => { + const cg = await indexed({ + 'table/table_test.cc': lines( + '#include ', + 'namespace rocksdb {', + 'class PinnedReader {', + ' public:', + ' void SetPinnedData(int data) {}', + '};', + 'class OtherReader {', + ' public:', + ' void SetPinnedData(int data) {}', + '};', + 'class PinnedFactory {', + ' public:', + ' PinnedReader* last_reader() const { return nullptr; }', + '};', + 'void Pin() {', + ' auto factory = std::make_shared();', + ' factory->last_reader()->SetPinnedData(1);', + '}', + 'void PinAgain(const std::unique_ptr& factory) {', + ' factory->last_reader()->SetPinnedData(2);', + '}', + '} // namespace rocksdb', + ), + }); + try { + expect(calls(cg, 'rocksdb::Pin')).toEqual(['rocksdb::PinnedFactory::last_reader @0.9', 'rocksdb::PinnedReader::SetPinnedData @0.85']); + expect(calls(cg, 'rocksdb::PinAgain')).toEqual(['rocksdb::PinnedFactory::last_reader @0.9', 'rocksdb::PinnedReader::SetPinnedData @0.85']); + } finally { + cg.close(); + } + }); + + it('the type of a local a chained call reads first is the same for its other calls', async () => { + // rocksdb's compaction test reads `cfd` while it works out a three-call + // chain on it; the local's type is kept for its other calls too. + const cg = await indexed({ + 'db/version_set.h': lines( + '#include ', + 'namespace rocksdb {', + 'class VersionStorageInfo {', + ' public:', + ' int LevelFiles(int level) const { return level; }', + '};', + 'class Version {', + ' public:', + ' VersionStorageInfo* storage_info() { return nullptr; }', + '};', + 'class ColumnFamilyData {', + ' public:', + ' Version* current() const { return nullptr; }', + '};', + 'class ColumnFamilySet {', + ' public:', + ' ColumnFamilyData* GetDefault() const { return nullptr; }', + '};', + 'class VersionSet {', + ' public:', + ' ColumnFamilySet* GetColumnFamilySet() { return nullptr; }', + '};', + '} // namespace rocksdb', + ), + 'db/compaction/compaction_job_test.cc': lines( + '#include "db/version_set.h"', + 'namespace rocksdb {', + 'class CompactionJobTestBase {', + ' public:', + ' void VerifyTables(int output_level);', + ' protected:', + ' std::unique_ptr versions_;', + '};', + 'void CompactionJobTestBase::VerifyTables(int output_level) {', + ' auto cfd = versions_->GetColumnFamilySet()->GetDefault();', + ' cfd->current()->storage_info()->LevelFiles(output_level);', + '}', + '} // namespace rocksdb', + ), + }); + try { + expect(calls(cg, 'rocksdb::CompactionJobTestBase::VerifyTables')).toEqual([ + 'rocksdb::ColumnFamilyData::current @0.9', + 'rocksdb::ColumnFamilySet::GetDefault @0.85', + 'rocksdb::Version::storage_info @0.85', + 'rocksdb::VersionSet::GetColumnFamilySet @0.9', + // The fork reads a chain of any length from the syntax tree (upstream reads one inner call), + // so the third call of the chain is typed too. + 'rocksdb::VersionStorageInfo::LevelFiles @0.85', + ]); + } finally { + cg.close(); + } + }); + + it('a chained call reaches the class the inner call is declared to return, not a namesake', async () => { + // protobuf's Java and C# generators each declare an EnumGenerator. + const cg = await indexed({ + 'src/google/protobuf/compiler/csharp/csharp_enum.h': lines( + 'namespace google {', + 'namespace protobuf {', + 'namespace compiler {', + 'namespace csharp {', + 'class EnumGenerator {', + ' public:', + ' void Generate(int printer) {}', + '};', + '} // namespace csharp', + '} // namespace compiler', + '} // namespace protobuf', + '} // namespace google', + ), + 'src/google/protobuf/compiler/java/generator_factory.h': lines( + 'namespace google {', + 'namespace protobuf {', + 'namespace compiler {', + 'namespace java {', + 'class EnumGenerator {', + ' public:', + ' void Generate(int printer) {}', + '};', + 'class GeneratorFactory {', + ' public:', + ' EnumGenerator* NewEnumGenerator(int descriptor) const { return nullptr; }', + '};', + '} // namespace java', + '} // namespace compiler', + '} // namespace protobuf', + '} // namespace google', + ), + 'src/google/protobuf/compiler/java/file.cc': lines( + '#include "src/google/protobuf/compiler/java/generator_factory.h"', + 'namespace google {', + 'namespace protobuf {', + 'namespace compiler {', + 'namespace java {', + 'void GenerateEnums(const GeneratorFactory* generator_factory) {', + ' generator_factory->NewEnumGenerator(1)->Generate(2);', + '}', + '} // namespace java', + '} // namespace compiler', + '} // namespace protobuf', + '} // namespace google', + ), + }); + try { + expect(calls(cg, 'google::protobuf::compiler::java::GenerateEnums')).toEqual([ + 'google::protobuf::compiler::java::EnumGenerator::Generate @0.85', + 'google::protobuf::compiler::java::GeneratorFactory::NewEnumGenerator @0.9', + ]); + } finally { + cg.close(); + } + }); +}); + +describe('what keeps the call as it was', () => { + it('an initializer that goes on past its line is read to its end', async () => { + // rocksdb's options test casts a member it reads on the line above the cast. + const cg = await indexed({ + 'include/rocksdb/merge_operator.h': lines( + '#include ', + 'namespace rocksdb {', + 'class Customizable {', + ' public:', + ' template ', + ' T* CheckedCast() { return nullptr; }', + '};', + 'class MergeOperator : public Customizable {', + ' public:', + ' virtual const char* Name() const { return ""; }', + '};', + 'struct ColumnFamilyOptions {', + ' std::shared_ptr merge_operator;', + '};', + 'namespace test {', + 'class ChanglingMergeOperator : public MergeOperator {', + ' public:', + ' void SetName(const char* name) {}', + '};', + '} // namespace test', + '} // namespace rocksdb', + ), + 'options/options_test.cc': lines( + '#include "include/rocksdb/merge_operator.h"', + 'namespace rocksdb {', + 'void Rename(ColumnFamilyOptions* base_cf_opt) {', + ' auto* merge_operator = base_cf_opt->merge_operator', + ' ->CheckedCast();', + ' merge_operator->SetName("other");', + '}', + '} // namespace rocksdb', + ), + }); + try { + // Not the shared_ptr member's type, which has no SetName. What `CheckedCast` + // returns is a template parameter, so the local's type is unknown and the fork, which guesses + // no method by the name of a receiver it cannot type, links nothing (upstream: SetName @0.7). + expect(calls(cg, 'rocksdb::Rename').filter((c) => c.includes('SetName'))).toEqual([]); + } finally { + cg.close(); + } + }); + + it('what a function template returns is a template parameter, after the macros that follow its header', async () => { + // protobuf's `Arena::Create` is declared `PROTOBUF_NDEBUG_INLINE static T* PROTOBUF_NONNULL Create(…)`. + const cg = await indexed({ + 'src/google/protobuf/arena.h': lines( + 'namespace google {', + 'namespace protobuf {', + 'class Arena {', + ' public:', + ' void Reset() {}', + ' template ', + ' PROTOBUF_FUTURE_ADD_EARLY_NODISCARD PROTOBUF_NDEBUG_INLINE static T*', + ' PROTOBUF_NONNULL', + ' Create(Arena* arena, Args&&... args) {', + ' return new T();', + ' }', + '};', + 'template ', + 'class RepeatedPtrField {', + ' public:', + ' Element* Add() { return nullptr; }', + '};', + 'class MessageLite {', + ' public:', + ' int AccessCachedSize() const { return 0; }', + '};', + 'class Message : public MessageLite {};', + 'class Pool {', + ' public:', + ' void Reset() {}', + '};', + 'PROTOBUF_EXPORT Arena* PROTOBUF_NONNULL DefaultArena() { return nullptr; }', + '// Overloads by argument: what each returns depends on T.', + 'template ', + 'PROTOBUF_FUTURE_ADD_EARLY_NODISCARD std::shared_ptr DownCastMessage(const std::shared_ptr& from) {', + ' return nullptr;', + '}', + 'template ', + 'PROTOBUF_FUTURE_ADD_EARLY_NODISCARD const T& DownCastMessage(const MessageLite& from) {', + ' return *static_cast(&from);', + '}', + '} // namespace protobuf', + '} // namespace google', + ), + 'src/google/protobuf/repeated_ptr_field_unittest.cc': lines( + '#include "src/google/protobuf/arena.h"', + 'namespace google {', + 'namespace protobuf {', + 'void Fill(Arena* arena) {', + ' auto* field = Arena::Create>(arena);', + ' field->Add();', + '}', + 'int ByteSizeLong(const MessageLite& msg) {', + ' auto& _this = DownCastMessage(msg);', + ' return _this.AccessCachedSize();', + '}', + 'void Clear() {', + ' auto* arena = DefaultArena();', + ' arena->Reset();', + '}', + '} // namespace protobuf', + '} // namespace google', + ), + }); + try { + // `T*` is a template parameter's type, which the local is left without: the fork guesses no + // method by the name of a receiver it cannot type (upstream: RepeatedPtrField::Add @0.7). + expect(calls(cg, 'google::protobuf::Fill').filter((c) => c.includes('::Add'))).toEqual([]); + // Not read as the first overload's `std::shared_ptr`, which would call a library method + // (upstream: MessageLite::AccessCachedSize @0.7). + expect(calls(cg, 'google::protobuf::ByteSizeLong').filter((c) => c.includes('AccessCachedSize'))).toEqual([]); + // A macro after the return type is no part of it. + expect(calls(cg, 'google::protobuf::Clear').filter((c) => c.includes('::Reset'))).toEqual(['google::protobuf::Arena::Reset @0.9']); + } finally { + cg.close(); + } + }); +}); + +describe('a chained call keeps the operator it was written with, and a loop variable its own type', () => { + it('`.` on a returned std::optional is the optional\'s member; `->` reaches what it holds', async () => { + const cg = await indexed({ + 'src/widget.cc': lines( + '#include ', + 'class Widget {', + ' public:', + ' int value() { return 1; }', + '};', + 'std::optional MakeWidget() { return std::nullopt; }', + 'void Dot() { MakeWidget().value(); }', + 'void Arrow() { MakeWidget()->value(); }', + ), + }); + try { + expect(calls(cg, 'Dot').filter((c) => c.includes('value'))).toEqual([]); + expect(calls(cg, 'Arrow').filter((c) => c.includes('value'))).toEqual(['Widget::value @0.85']); + } finally { + cg.close(); + } + }); + + it('an `auto` or declared variable of a `for` initializer shadows a member of a base class', async () => { + const cg = await indexed({ + 'src/loop.cc': lines( + 'class Widget {', + ' public:', + ' void spin() {}', + '};', + 'class Gadget {', + ' public:', + ' void spin() {}', + '};', + 'Gadget* MakeGadget() { return nullptr; }', + 'class Base {', + ' protected:', + ' Widget* cursor;', + '};', + 'class Derived : public Base {', + ' public:', + ' void Run() {', + ' for (auto cursor = MakeGadget(); cursor != nullptr; cursor = nullptr) cursor->spin();', + ' }', + ' void RunDeclared() {', + ' for (Gadget* cursor = MakeGadget(); cursor != nullptr; cursor = nullptr) cursor->spin();', + ' }', + '};', + ), + }); + try { + expect(calls(cg, 'Derived::Run').filter((c) => c.includes('spin'))).toEqual(['Gadget::spin @0.9']); + expect(calls(cg, 'Derived::RunDeclared').filter((c) => c.includes('spin'))).toEqual(['Gadget::spin @0.9']); + } finally { + cg.close(); + } + }); +}); + +describe('a type from outside the project reaches none of its classes', () => { + it('one under a namespace the project declares nothing in', async () => { + const cg = await indexed({ + 'upb/status.h': lines( + 'namespace upb {', + 'class Status {', + ' public:', + ' bool ok() const { return true; }', + '};', + '} // namespace upb', + ), + 'conformance/conformance_test.cc': lines( + '#include "absl/status/status.h"', + '#include "upb/status.h"', + 'namespace google {', + 'namespace protobuf {', + 'bool Passed(const absl::Status& status) {', + ' return status.ok();', + '}', + 'absl::Status Parse(int payload) { return absl::Status(); }', + 'bool Parsed(int payload) {', + ' return Parse(payload).ok();', + '}', + '} // namespace protobuf', + '} // namespace google', + ), + // A specialization the index reads as a method named `google::protobuf::absl::Cord`. + 'src/google/protobuf/lite_unittest.cc': lines( + '#include "absl/strings/cord.h"', + 'namespace google {', + 'namespace protobuf {', + 'template ', + 'T SerializeAs(const int& msg);', + 'template <>', + 'absl::Cord SerializeAs(const int& msg) {', + ' return absl::Cord();', + '}', + '} // namespace protobuf', + '} // namespace google', + ), + }); + try { + expect(calls(cg, 'google::protobuf::Passed')).toEqual([]); + // What a function returns, for a chained call. + expect(calls(cg, 'google::protobuf::Parsed').filter((c) => c.includes('::ok'))).toEqual([]); + } finally { + cg.close(); + } + }); + + it("a standard one whose name a class of the project shares", async () => { + // fmt opens its namespace with a macro and declares an `ostream` of its own. + const cg = await indexed({ + 'include/fmt/os.h': lines( + '#define FMT_BEGIN_NAMESPACE namespace fmt { inline namespace v11 {', + '#define FMT_END_NAMESPACE } }', + 'FMT_BEGIN_NAMESPACE', + 'class ostream {', + ' public:', + ' void flush() {}', + '};', + 'FMT_END_NAMESPACE', + ), + 'src/ostream.cc': lines( + '#include ', + '#include "include/fmt/os.h"', + 'FMT_BEGIN_NAMESPACE', + 'void flush_stream(std::ostream& os) {', + ' os.flush();', + '}', + 'FMT_END_NAMESPACE', + ), + }); + try { + expect(calls(cg, 'flush_stream')).toEqual([]); + } finally { + cg.close(); + } + }); + + it('not one of a namespace the project declares, which the file does not see', async () => { + const cg = await indexed({ + 'src/google/protobuf/descriptor.pb.h': lines( + 'namespace google {', + 'namespace protobuf {', + 'class FileDescriptorProto {', + ' public:', + ' void Clear() {}', + '};', + '} // namespace protobuf', + '} // namespace google', + ), + 'src/google/protobuf/compiler/java/shared_code_generator.cc': lines( + 'namespace google {', + 'namespace protobuf {', + 'void Reset(google::protobuf::FileDescriptorProto& file_proto) {', + ' file_proto.Clear();', + '}', + '} // namespace protobuf', + '} // namespace google', + ), + }); + try { + expect(calls(cg, 'google::protobuf::Reset')).toEqual(['google::protobuf::FileDescriptorProto::Clear @0.9']); + } finally { + cg.close(); + } + }); +}); diff --git a/__tests__/cpp-member-receiver.test.ts b/__tests__/cpp-member-receiver.test.ts new file mode 100644 index 0000000000..47361e7da6 --- /dev/null +++ b/__tests__/cpp-member-receiver.test.ts @@ -0,0 +1,516 @@ +/** + * A C++ call on a data member of the calling method's class reaches the + * method of the member's type, wherever the class is declared. + * + * Receiver inference reads back through the call's file, then the header of + * the same name (`foo.cc` → `foo.h`). In facebook/rocksdb, `DBImpl` is + * declared in db_impl.h and its methods are defined in db_impl_write.cc, + * db_impl_compaction_flush.cc, db_impl_open.cc and more, so `versions_`, + * `mutex_` and the rest were never read there: their calls were guessed by + * the receiver's name. Neither was a member declared below an inline method + * that uses it, or one a base class declares. A gtest `TEST_F(Fixture, …)` + * body is a method of a class derived from `Fixture`, so `db_` there is the + * fixture's member. + * + * C++ looks a name up in the function first, then in its class and the + * classes it derives from: a parameter or local of the name, or one the scan + * can't read, still decides. A guess can land on the right method by a shared + * word, so these tests check the confidence, not only the target. + */ +import { describe, it, expect, afterEach } from 'vitest'; +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import { CodeGraph } from '../src'; +import type { Node } from '../src/types'; + +const roots: string[] = []; +afterEach(() => { + for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true }); +}); + +async function indexed(files: Record): Promise { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cpp-member-')); + roots.push(root); + for (const [rel, content] of Object.entries(files)) { + fs.mkdirSync(path.dirname(path.join(root, rel)), { recursive: true }); + fs.writeFileSync(path.join(root, rel), content); + } + return CodeGraph.init(root, { index: true }); +} + +function callable(cg: CodeGraph, qualifiedName: string): Node { + const node = [...cg.getNodesByKind('method'), ...cg.getNodesByKind('function')].find((n) => n.qualifiedName === qualifiedName); + if (!node) throw new Error(`no function or method ${qualifiedName}`); + return node; +} + +/** `calls` callees of a function or method, as `qualifiedName @confidence`, one per call site. */ +function calls(cg: CodeGraph, caller: string): string[] { + return cg + .getCallees(callable(cg, caller).id) + .filter((r) => r.edge.kind === 'calls') + .map((r) => `${r.node.qualifiedName} @${(r.edge.metadata as { confidence?: number } | undefined)?.confidence}`) + .sort(); +} + +const lines = (...text: string[]): string => [...text, ''].join('\n'); + +/** rocksdb's DBImpl, trimmed: declared in db_impl.h, its methods defined in other files. */ +const ROCKSDB = { + 'db/version_set.h': lines( + 'namespace rocksdb {', + 'class ColumnFamilySet {', + ' public:', + ' int NumberOfColumnFamilies() const { return 0; }', + '};', + 'class VersionSet {', + ' public:', + ' ColumnFamilySet* GetColumnFamilySet() { return nullptr; }', + ' void SetLastSequence(int s) {}', + '};', + 'class ColumnFamilyHandleImpl {', + ' public:', + ' ColumnFamilySet* GetColumnFamilySet() { return nullptr; }', + ' void SetLastSequence(int s) {}', + '};', + 'class Statistics {', + ' public:', + ' bool load() const { return false; }', + '};', + '} // namespace rocksdb', + ), + 'monitoring/instrumented_mutex.h': lines( + 'namespace rocksdb {', + 'class InstrumentedMutex {', + ' public:', + ' void Lock() {}', + ' void Unlock() {}', + '};', + 'namespace port {', + 'class Mutex {', + ' public:', + ' void Lock() {}', + ' void Unlock() {}', + '};', + '} // namespace port', + '} // namespace rocksdb', + ), + 'db/db_impl/db_impl.h': lines( + '#include ', + '#include ', + '#include "db/version_set.h"', + '#include "monitoring/instrumented_mutex.h"', + 'namespace rocksdb {', + 'class DBImpl {', + ' public:', + ' void WriteImpl();', + ' void FlushMemTable();', + ' protected:', + ' std::unique_ptr versions_;', + ' InstrumentedMutex mutex_;', + ' std::atomic shutting_down_;', + '};', + '} // namespace rocksdb', + ), +}; + +describe('a data member of the calling method’s class', () => { + it('reproduction: one the class declares in another header', async () => { + const cg = await indexed({ + ...ROCKSDB, + 'db/db_impl/db_impl_write.cc': lines( + '#include "db/db_impl/db_impl.h"', + 'namespace rocksdb {', + '// Another function\'s local of the name, which the method does not see.', + 'void UnlockStale(port::Mutex* mutex_) { mutex_->Unlock(); }', + 'void DBImpl::WriteImpl() {', + ' mutex_.Lock();', + ' versions_->SetLastSequence(1);', + ' versions_->GetColumnFamilySet()->NumberOfColumnFamilies();', + ' if (shutting_down_.load()) return;', + ' mutex_.Unlock();', + '}', + '} // namespace rocksdb', + ), + }); + try { + // A `std::atomic` member calls no method of the project's. + expect(calls(cg, 'rocksdb::DBImpl::WriteImpl')).toEqual([ + 'rocksdb::ColumnFamilySet::NumberOfColumnFamilies @0.85', + 'rocksdb::InstrumentedMutex::Lock @0.9', + 'rocksdb::InstrumentedMutex::Unlock @0.9', + 'rocksdb::VersionSet::GetColumnFamilySet @0.9', + 'rocksdb::VersionSet::SetLastSequence @0.9', + ]); + } finally { + cg.close(); + } + }); + + it('one declared below the inline method that uses it, and one a base class declares', async () => { + const cg = await indexed({ + ...ROCKSDB, + 'db/memtable_list.h': lines( + 'namespace rocksdb {', + 'class MemTableListVersion {', + ' public:', + ' int NumNotFlushed() const { return 0; }', + '};', + 'class MemTable {', + ' public:', + ' int NumNotFlushed() const { return 1; }', + '};', + 'class MemTableList {', + ' public:', + ' bool IsFlushPending() const { return current_->NumNotFlushed() > 0; }', + ' // A line that starts with a dereference is code, not a comment.', + ' int Pick(const int* first) const {', + ' return Choose(0,', + ' *first);', + ' }', + ' private:', + ' int Choose(int a, int b) const { return a + b; }', + ' /*', + ' MemTable* current_;', + ' */', + ' MemTableListVersion* current_;', + '};', + '} // namespace rocksdb', + ), + 'db/db_impl/db_impl_secondary.h': lines( + '#include "db/db_impl/db_impl.h"', + 'namespace rocksdb {', + 'class DBImplSecondary : public DBImpl {', + ' public:', + ' void TryCatchUpWithPrimary();', + '};', + '} // namespace rocksdb', + ), + 'db/db_impl/db_impl_secondary.cc': lines( + '#include "db/db_impl/db_impl_secondary.h"', + 'namespace rocksdb {', + 'void DBImplSecondary::TryCatchUpWithPrimary() {', + ' versions_->SetLastSequence(2);', + '}', + '} // namespace rocksdb', + ), + }); + try { + expect(calls(cg, 'rocksdb::MemTableList::IsFlushPending')).toEqual(['rocksdb::MemTableListVersion::NumNotFlushed @0.9']); + expect(calls(cg, 'rocksdb::DBImplSecondary::TryCatchUpWithPrimary')).toEqual(['rocksdb::VersionSet::SetLastSequence @0.9']); + } finally { + cg.close(); + } + }); + + it("a gtest TEST_F body's fixture member", async () => { + const cg = await indexed({ + 'include/rocksdb/iterator.h': lines( + 'namespace rocksdb {', + 'class IteratorBase {', + ' public:', + ' virtual void SeekToFirst() = 0;', + '};', + 'class Iterator : public IteratorBase {};', + 'class MemTableRep {', + ' public:', + ' class Iterator {', + ' public:', + ' virtual void SeekToFirst() = 0;', + ' };', + '};', + 'class DB {', + ' public:', + ' virtual Iterator* NewIterator() = 0;', + '};', + '} // namespace rocksdb', + ), + 'db/db_test_util.h': lines( + '#include "include/rocksdb/iterator.h"', + 'namespace rocksdb {', + 'class DBTestBase {', + ' protected:', + ' DB* db_;', + '};', + '} // namespace rocksdb', + ), + 'db/db_iterator_test.cc': lines( + '#include "db/db_test_util.h"', + 'namespace rocksdb {', + 'class DBIteratorTest : public DBTestBase {};', + 'TEST_F(DBIteratorTest, IterSeekToFirst) {', + ' auto iter = db_->NewIterator();', + ' iter->SeekToFirst();', + '}', + '} // namespace rocksdb', + ), + }); + try { + expect(calls(cg, 'rocksdb::TEST_F')).toEqual(['rocksdb::DB::NewIterator @0.9', 'rocksdb::IteratorBase::SeekToFirst @0.9']); + } finally { + cg.close(); + } + }); +}); + +describe('a data member of the class itself', () => { + it('not one of a class nested in it, which its header declares first', async () => { + // rocksdb's `MultiScan::MultiScanIterator` has a `scan_opts_` of its own. + const cg = await indexed({ + 'table/multi_scan.h': lines( + 'namespace rocksdb {', + 'class ScanOptions {', + ' public:', + ' int size() const { return 0; }', + '};', + 'class ScanRanges {', + ' public:', + ' int size() const { return 1; }', + '};', + 'class MultiScan {', + ' public:', + ' class MultiScanIterator {', + ' private:', + ' ScanRanges* scan_opts_;', + ' };', + ' int Count() const;', + ' private:', + ' ScanOptions* scan_opts_;', + '};', + '} // namespace rocksdb', + ), + 'table/multi_scan.cc': lines( + '#include "table/multi_scan.h"', + 'namespace rocksdb {', + 'int MultiScan::Count() const { return scan_opts_->size(); }', + '} // namespace rocksdb', + ), + }); + try { + expect(calls(cg, 'rocksdb::MultiScan::Count')).toEqual(['rocksdb::ScanOptions::size @0.9']); + } finally { + cg.close(); + } + }); +}); + +describe('which class declares the member', () => { + it('the declaration in the caller’s own translation unit, of classes that share a name', async () => { + // google/leveldb's memenv.cc and fault_injection_test.cc each declare a `FileState`. + const cg = await indexed({ + 'include/leveldb/slice.h': lines( + 'namespace leveldb {', + 'class Slice {', + ' public:', + ' int size() const { return 0; }', + '};', + 'class Block {', + ' public:', + ' int size() const { return 1; }', + '};', + '} // namespace leveldb', + ), + 'helpers/memenv/memenv.cc': lines( + '#include "include/leveldb/slice.h"', + 'namespace leveldb {', + 'class FileState {', + ' public:', + ' int Size() const;', + ' private:', + ' Slice* blocks_;', + '};', + 'int FileState::Size() const { return blocks_->size(); }', + '} // namespace leveldb', + ), + 'db/fault_injection_test.cc': lines( + '#include "include/leveldb/slice.h"', + 'namespace leveldb {', + 'class FileState {', + ' public:', + ' int Size() const;', + ' private:', + ' Block* blocks_;', + '};', + 'int FileState::Size() const { return blocks_->size(); }', + '} // namespace leveldb', + ), + }); + try { + const sizes = cg.getNodesByKind('method').filter((n) => n.qualifiedName === 'leveldb::FileState::Size'); + const byFile = Object.fromEntries(sizes.map((n) => [n.filePath, cg.getCallees(n.id).filter((r) => r.edge.kind === 'calls').map((r) => r.node.qualifiedName)])); + expect(byFile).toEqual({ + 'db/fault_injection_test.cc': ['leveldb::Block::size'], + 'helpers/memenv/memenv.cc': ['leveldb::Slice::size'], + }); + } finally { + cg.close(); + } + }); + + it('not a base that depends on the template’s parameters, where a bare name is not looked up', async () => { + const cg = await indexed({ + ...ROCKSDB, + 'util/heap_test.cc': lines( + '#include "db/db_impl/db_impl.h"', + 'namespace rocksdb {', + 'template ', + 'class Holder {', + ' protected:', + ' InstrumentedMutex mutex_;', + '};', + 'port::Mutex mutex_;', + 'template ', + 'class Guard : public Holder {', + ' public:', + ' void Enter() { mutex_.Lock(); }', + '};', + '} // namespace rocksdb', + ), + }); + try { + // C++ finds the namespace's `mutex_` here: `this->mutex_` would name the base's. + expect(calls(cg, 'rocksdb::Guard::Enter')).not.toContain('rocksdb::InstrumentedMutex::Lock @0.9'); + } finally { + cg.close(); + } + }); +}); + +describe('what keeps the call as it was', () => { + it('a local of the name that the scan cannot read', async () => { + const cg = await indexed({ + ...ROCKSDB, + 'db/db_impl/db_impl_compaction_flush.cc': lines( + '#include "db/db_impl/db_impl.h"', + 'namespace rocksdb {', + 'void DBImpl::FlushMemTable() {', + ' auto mutex_ = GetFlushMutex();', + ' mutex_->Lock();', + '}', + '} // namespace rocksdb', + ), + }); + try { + // `mutex_` here is the local, not the member. + expect(calls(cg, 'rocksdb::DBImpl::FlushMemTable').filter((c) => c.includes('Lock'))).not.toContain('rocksdb::InstrumentedMutex::Lock @0.9'); + } finally { + cg.close(); + } + }); + + it('a parameter of a method the class declares, which is no member', async () => { + // rocksdb's DB declares `CreateColumnFamilies(const std::vector<…>& + // column_families, …)`; DBImpl's `const ColumnFamilySet* const + // column_families` is a local the scan cannot read. + const cg = await indexed({ + ...ROCKSDB, + 'db/db_impl/db_impl.h': ROCKSDB['db/db_impl/db_impl.h'].replace( + ' void FlushMemTable();', + ' void FlushMemTable();\n void CreateColumnFamilies(std::vector* column_families,\n int* handles);\n int CountColumnFamilies();\n ColumnFamilySet* stats() const;', + ), + 'db/db_impl/db_impl_write.cc': lines( + '#include "db/db_impl/db_impl.h"', + 'namespace rocksdb {', + 'int DBImpl::CountColumnFamilies() {', + ' const ColumnFamilySet* const column_families = versions_->GetColumnFamilySet();', + ' const Statistics* const stats = nullptr;', + ' if (stats->load()) return 0;', + ' return column_families->NumberOfColumnFamilies();', + '}', + '} // namespace rocksdb', + ), + }); + try { + // The fork reads the local's declared type from the syntax tree, so these are typed calls + // (upstream's line scan cannot read `const ColumnFamilySet* const` and guesses, at 0.7). + expect(calls(cg, 'rocksdb::DBImpl::CountColumnFamilies').filter((c) => c.includes('NumberOfColumnFamilies'))).toEqual([ + 'rocksdb::ColumnFamilySet::NumberOfColumnFamilies @0.9', + ]); + // Nor is a method the class declares (`stats()`) a member of its name. + expect(calls(cg, 'rocksdb::DBImpl::CountColumnFamilies').filter((c) => c.includes('load'))).toEqual(['rocksdb::Statistics::load @0.9']); + } finally { + cg.close(); + } + }); + + it("a member of a template parameter's type", async () => { + // rocksdb's `ComparatorWithU64TsImpl` holds a `TComparator cmp_without_ts_`. + const cg = await indexed({ + 'util/comparator.cc': lines( + 'namespace rocksdb {', + 'class Comparator {', + ' public:', + ' int timestamp_size() const { return 0; }', + '};', + 'template ', + 'class ComparatorWithU64TsImpl {', + ' public:', + ' int Check() const { return cmp_without_ts_.timestamp_size(); }', + ' private:', + ' TComparator cmp_without_ts_;', + '};', + '// rocksdb\'s clock cache test reaches its shard through a dependent alias.', + 'class ClockCacheShard {', + ' public:', + ' void Insert(int key) {}', + '};', + 'template ', + 'class ClockCacheTest {', + ' public:', + ' using Shard = typename ClockCache::Shard;', + ' void Insert(int key) { shard_->Insert(key); }', + ' private:', + ' Shard* shard_ = nullptr;', + '};', + '} // namespace rocksdb', + ), + }); + try { + // The member's type is a template parameter, which the fork does not guess a method of by the + // receiver's name (upstream: Comparator::timestamp_size @0.7, ClockCacheShard::Insert @0.65). + expect(calls(cg, 'rocksdb::ComparatorWithU64TsImpl::Check')).toEqual([]); + expect(calls(cg, 'rocksdb::ClockCacheTest::Insert')).toEqual([]); + } finally { + cg.close(); + } + }); + + it('a member a base class template declares through an alias of its parameter', async () => { + // protobuf's MapIteratorBase holds a `MapFieldBase* map_` through + // `using MapFieldBase = std::conditional_t`. + const cg = await indexed({ + 'src/google/protobuf/map_field.h': lines( + '#include ', + 'namespace google {', + 'namespace protobuf {', + 'namespace internal {', + 'class MapFieldBase {', + ' public:', + ' void SetMapDirty() {}', + '};', + '} // namespace internal', + 'template ', + 'class MapIteratorBase {', + ' protected:', + ' using MapFieldBase = std::conditional_t;', + ' MapFieldBase* map_;', + '};', + 'class MapIterator : public MapIteratorBase {', + ' public:', + ' void MutableValueRef() { map_->SetMapDirty(); }', + '};', + '} // namespace protobuf', + '} // namespace google', + ), + 'src/google/protobuf/map_field.cc': lines('#include "src/google/protobuf/map_field.h"'), + }); + try { + // The member's type is an alias of a trait of the base's parameter, which no lookup places; the + // fork guesses no method by the receiver's name (upstream: MapFieldBase::SetMapDirty @0.9). + expect(calls(cg, 'google::protobuf::MapIterator::MutableValueRef')).toEqual([]); + } finally { + cg.close(); + } + }); +}); diff --git a/codegraph-kernel/src/resolve/cpp_implicit_this.rs b/codegraph-kernel/src/resolve/cpp_implicit_this.rs index 62a9087b11..e88091350c 100644 --- a/codegraph-kernel/src/resolve/cpp_implicit_this.rs +++ b/codegraph-kernel/src/resolve/cpp_implicit_this.rs @@ -91,7 +91,7 @@ fn argument_list_follows(text: &str, from: usize) -> bool { } /// stripCppTemplateArguments: the text with every `<…>` span removed. -fn strip_template_arguments(text: &str) -> String { +pub(super) fn strip_template_arguments(text: &str) -> String { let mut out = String::new(); let mut depth = 0usize; for c in text.chars() { @@ -105,7 +105,7 @@ fn strip_template_arguments(text: &str) -> String { out } -fn contains_word(text: &str, word: &str) -> bool { +pub(super) fn contains_word(text: &str, word: &str) -> bool { text.match_indices(word).any(|(at, _)| { !text[..at].chars().next_back().is_some_and(word_char) && !text[at + word.len()..].chars().next().is_some_and(word_char) }) @@ -157,7 +157,7 @@ impl KernelResolver { at(column) } - fn cpp_class_decls(&mut self, qualified_name: &str) -> Res>> { + pub(super) fn cpp_class_decls(&mut self, qualified_name: &str) -> Res>> { Ok(self.nodes_by_qualified_name(qualified_name)?.iter().filter(|n| class_kind(&n.kind) && c_family(&n.language)).cloned().collect()) } @@ -382,7 +382,7 @@ impl KernelResolver { /// Whether `base`, a base of the C++ class declaration `decl`, depends on the template /// parameters of `decl` or a class around it: `template class Foo : public Base`. /// A bare name is not looked up in such a base. - fn cpp_dependent_base(&mut self, decl: &KNode, base: &KNode) -> Res { + pub(super) fn cpp_dependent_base(&mut self, decl: &KNode, base: &KNode) -> Res { let parameters = self.cpp_template_parameter_names(decl)?; if parameters.is_empty() { return Ok(false); } let spec = self.cpp_base_specifiers(decl).into_iter().find(|s| cpp_last_segment(strip_template_arguments(s).trim()) == base.name); @@ -391,7 +391,7 @@ impl KernelResolver { /// The template parameters in scope where `node` is declared: its own `template <…>` header /// and those of the class templates around it. - fn cpp_template_parameter_names(&mut self, node: &KNode) -> Res> { + pub(super) fn cpp_template_parameter_names(&mut self, node: &KNode) -> Res> { let mut names = HashSet::new(); let Some(source) = self.read_file(&node.file_path) else { return Ok(names) }; collect_template_header(&source, node.start_line, node.start_column, &mut names); @@ -440,7 +440,9 @@ fn collect_template_header(source: &SourceFile, start_line: i64, start_column: i before.push(¤t[..names::js_unit_to_byte(current, start_column.max(0) as usize)]); let before = before.join("\n"); let Some(at) = before.rfind("template") else { return }; - let Some(header) = re!(r"(?s)^template\s*<(.*)>\s*$").captures(&before[at..]) else { return }; + // Specifiers, attributes and macros may come between the header and where the declaration + // starts: protobuf's `template ` / `PROTOBUF_NDEBUG_INLINE static T* Create(…)`. + let Some(header) = re!(r"(?s)^template\s*<(.*)>(?:\s*(?:\[\[[^\]]*\]\]|(?:static|inline|constexpr|consteval|virtual|explicit|friend|extern)\b|[A-Z][A-Z0-9]*_[A-Z0-9_]*\b(?:\s*\([^()]*\))?))*\s*$").captures(&before[at..]) else { return }; for item in split_top_level(&header[1]) { // `typename T = int`, `template class Policy`, `typename... Ts`, `int N`. let stripped = strip_template_arguments(&item); diff --git a/codegraph-kernel/src/resolve/cpp_namespace.rs b/codegraph-kernel/src/resolve/cpp_namespace.rs index c924a48bdb..7c76363130 100644 --- a/codegraph-kernel/src/resolve/cpp_namespace.rs +++ b/codegraph-kernel/src/resolve/cpp_namespace.rs @@ -49,6 +49,26 @@ impl KernelResolver { let values:HashSet<_>=self.cpp_namespaces.replacements.iter().filter(|((f,n),_)|visible.contains(f) && n==name).map(|(_,v)|v.clone()).collect(); Ok(if values.len()==1 {values.into_iter().next().unwrap()} else {name.to_string()}) } + /// cppProjectScopeNames (name-matcher.ts): every name a qualifier may start with from some + /// scope in this project — each segment of the qualified names of the types its C and C++ code + /// declares, the outermost namespaces its namespace-opening macros open (`fmt` for + /// `FMT_BEGIN_NAMESPACE`, which no qualified name shows), and its namespace aliases. Not the + /// names of functions: protobuf's conversion operators read as methods named + /// `google::protobuf::absl::Cord`. + pub(super) fn cpp_project_scope_names(&mut self) -> Res>> { + if let Some(hit) = &self.cpp_scope_names { return Ok(hit.clone()); } + self.prepare_namespace_macros()?; + let mut names: HashSet = self.cpp_namespaces.openers.values().filter_map(|path| path.first().cloned()).collect(); + names.extend(self.cpp_namespaces.aliases.values().flat_map(|aliases| aliases.keys().cloned())); + for node in self.table()?.nodes.iter() { + if matches!(node.language.as_str(), "c" | "cpp") && matches!(node.kind.as_str(), "class" | "struct" | "union" | "enum" | "type_alias") { + names.extend(node.qualified_name.split("::").filter(|s| !s.is_empty()).map(str::to_string)); + } + } + let names = Rc::new(names); + self.cpp_scope_names = Some(names.clone()); + Ok(names) + } pub(super) fn namespace_frames(&mut self,file:&str)->Res)>> { self.prepare_namespace_macros()?; if let Some(hit)=self.cpp_namespaces.frames.get(file) {return Ok(hit.clone());} diff --git a/codegraph-kernel/src/resolve/cpp_range_for.rs b/codegraph-kernel/src/resolve/cpp_range_for.rs index 8d19c5f3f0..513c6e53c3 100644 --- a/codegraph-kernel/src/resolve/cpp_range_for.rs +++ b/codegraph-kernel/src/resolve/cpp_range_for.rs @@ -16,6 +16,7 @@ //! looked up where it is written. A type a template parameter decides, or one that can't be read, //! gives no element, and the call resolves as it did before. use super::cpp_aliases::type_segments; +use super::cpp_implicit_this::strip_template_arguments; use super::cpp_receivers::{cpp_ast_declared_type, cpp_binding_type, cpp_declarator_name, cpp_held_type}; use super::iteration::{descendant_for_position, named_children}; use super::*; @@ -128,9 +129,13 @@ fn without_spaces(text: &str) -> String { text.chars().filter(|c| !c.is_whitespa /// The initializer of `name`'s declaration in the block `parent`, before `at`: `vstorage-> /// GetBlobFiles()` in `const auto& blob_files = vstorage->GetBlobFiles();`. The same declarations /// `cpp_binding_type` reads. -fn binding_initializer<'t>(parent: TsNode<'t>, at: TsNode<'t>, name: &str, source: &SourceFile) -> Option> { - if !matches!(parent.kind(), "compound_statement" | "declaration_list" | "translation_unit") { return None; } - for declaration in named_children(parent).into_iter().rev().filter(|n| n.kind() == "declaration" && n.end_byte() <= at.start_byte()) { +pub(super) fn binding_initializer<'t>(parent: TsNode<'t>, at: TsNode<'t>, name: &str, source: &SourceFile) -> Option> { + let children = match parent.kind() { + "compound_statement" | "declaration_list" | "translation_unit" => named_children(parent), + _ => statement_initializer(parent).into_iter().collect(), + }; + if children.is_empty() { return None; } + for declaration in children.into_iter().rev().filter(|n| n.kind() == "declaration" && n.end_byte() <= at.start_byte()) { let mut cursor = declaration.walk(); let declarators: Vec<_> = declaration.children_by_field_name("declarator", &mut cursor).collect(); if let Some(declarator) = declarators.into_iter().find(|d| cpp_declarator_name(*d, source) == name) { @@ -141,6 +146,46 @@ fn binding_initializer<'t>(parent: TsNode<'t>, at: TsNode<'t>, name: &str, sourc None } +/// The declaration a `for`, `if`, `switch` or `while` statement (or its condition) opens with, +/// which names variables for the statement as a block's declarations do for the block. +pub(super) fn statement_initializer(node: TsNode<'_>) -> Option> { + match node.kind() { + "for_statement" | "condition_clause" => node.child_by_field_name("initializer"), + "if_statement" | "switch_statement" | "while_statement" => node.child_by_field_name("condition").and_then(|c| c.child_by_field_name("initializer")), + _ => None, + }.filter(|n| n.kind() == "declaration") +} + +/// Whether the written type `text` is a template parameter of `parameters`, or a library template +/// of one (`std::shared_ptr`, `std::vector`). +fn type_depends_on(text: &str, parameters: &HashSet, depth: u32) -> bool { + if depth > 3 { return false; } + let plain = cpp_plain_type(text); + let primary = plain.split('<').next().unwrap_or("").trim().trim_start_matches("::").trim_end_matches(['*', '&', ' ']); + if parameters.contains(primary.split("::").next().unwrap_or("").trim()) { return true; } + if let Some(held) = re!(r"^(?:::)?std::(?:unique_ptr|shared_ptr|optional)\s*<(.*)>[\s*&]*$").captures(&plain).and_then(|c| cpp_held_type(&c[1])) { + return type_depends_on(&held, parameters, depth + 1); + } + re!(r"^(?:::)?(?:std|absl)::").is_match(&plain) && re!(r"[A-Za-z_]\w*").find_iter(&plain).any(|word| parameters.contains(word.as_str())) +} + +/// A call expression's text as the extractor spells a chain: no spaces, `.` for `->`, and the +/// arguments of each call left out (`factory.NewEnumGenerator()`). +fn chain_spelling(text: &str) -> String { + let text: String = text.split_whitespace().collect::().replace("->", "."); + let mut out = String::new(); + let mut depth = 0usize; + for c in text.chars() { + match c { + '(' => { if depth == 0 { out.push('('); } depth += 1; } + ')' => { depth = depth.saturating_sub(1); if depth == 0 { out.push(')'); } } + _ if depth == 0 => out.push(c), + _ => {} + } + } + out +} + fn site_at(r: &ResolveRefIn, node: &KNode) -> ResolveRefIn { let mut site = r.clone().at(node); site.from_node_id = node.id.clone(); @@ -291,18 +336,162 @@ impl KernelResolver { Ok(Some((raw, parameters))) } - /// The type the C++ class `cls` declares its data member `field` with, as written in the class. - fn cpp_data_member_written(&mut self, cls: &str, field: &str, r: &ResolveRefIn) -> Res> { + /// The type the C++ class `cls` declares its data member `field` with, as written in the class, + /// or in the nearest class it derives from that declares it. A base that depends on the + /// template's parameters is not looked in: a bare name is not looked up there. + pub(super) fn cpp_data_member_written(&mut self, cls: &str, field: &str, r: &ResolveRefIn) -> Res> { + self.cpp_member_written_in(cls, field, r, 0, &mut HashSet::new()) + } + + fn cpp_member_written_in(&mut self, cls: &str, field: &str, r: &ResolveRefIn, depth: usize, seen: &mut HashSet) -> Res> { + if !seen.insert(cls.to_string()) { return Ok(None); } let visible = self.namespace_visible_files(&r.file_path, "cpp")?; let decls: Vec<_> = self.nodes_by_qualified_name(cls)?.iter().filter(|n| cpp_class_node(n) && visible.contains(&n.file_path)).cloned().collect(); - for decl in decls { - if let Some(text) = self.cpp_field_type(&decl, field, r)? { - return Ok(Some(CppWritten { text, site: site_at(r, &decl) })); + // Classes that share a name each write their own: the first one that declares the member + // decides, unless another writes it differently. + let mut found: Option = None; + for decl in &decls { + match self.cpp_field_types(decl, field, r)?.as_slice() { + [] => {} + [text] => match &found { + None => found = Some(CppWritten { text: text.clone(), site: site_at(r, decl) }), + Some(first) if without_spaces(&first.text) == without_spaces(text) => {} + Some(_) => return Ok(None), + }, + // Declared more than once: not one of them is the member's. + _ => return Ok(None), + } + } + if found.is_some() { return Ok(found); } + if depth >= 4 || !self.supertypes_complete { return Ok(None); } + for decl in &decls { + for base in self.supertype_nodes(&decl.id)? { + if !cpp_class_node(&base) || self.cpp_dependent_base(decl, &base)? { continue; } + if let Some(member) = self.cpp_member_written_in(&base.qualified_name, field, r, depth + 1, seen)? { return Ok(Some(member)); } } } Ok(None) } + /// The data member `name` of the class the caller's body is in, or of its gtest fixture: where + /// the scan leaves the calling function, C++ looks a name up in its class and the classes that + /// class derives from. + pub(super) fn cpp_caller_member(&mut self, name: &str, r: &ResolveRefIn) -> Res> { + let Some(caller) = self.node_by_id(&r.from_node_id)? else { return Ok(None) }; + let owner = match self.cpp_member_owner(&caller)? { + Some(owner) => Some(owner), + None => self.cpp_gtest_fixture(&caller, r)?, + }; + match owner { + Some(owner) => self.cpp_data_member_written(&owner, name, r), + None => Ok(None), + } + } + + /// The fixture class of a gtest `TEST_F(Fixture, Name)` / `TEST_P(Fixture, Name)` body, which is + /// a method of a class derived from it. + fn cpp_gtest_fixture(&mut self, caller: &KNode, r: &ResolveRefIn) -> Res> { + if !matches!(caller.name.as_str(), "TEST_F" | "TEST_P") || !matches!(caller.kind.as_str(), "function" | "method") { return Ok(None); } + let Some(source) = self.read_file(&caller.file_path) else { return Ok(None) }; + let from = (caller.start_line - 1).max(0) as usize; + let head = (from..from + 3).filter_map(|i| source.get(i).map(String::as_str)).collect::>().join(" "); + let Some(fixture) = re!(r"\bTEST_[FP]\s*\(\s*((?:::)?[A-Za-z_]\w*(?:::[A-Za-z_]\w*)*)\s*,").captures(&head).map(|m| m[1].to_string()) else { return Ok(None) }; + Ok(self.cpp_type_owner(&fixture, r, 0, false)?.filter(|n| cpp_class_node(n)).map(|n| n.qualified_name.clone())) + } + + /// Whether `written` is a template parameter's type where it is written (`T*`, `const T&`), or + /// a library template of one (`std::shared_ptr`, `std::vector`), whose members are the + /// parameter's. A class template of the project's is that class, whatever its arguments: + /// `std::unique_ptr>` holds one. + fn cpp_written_template_dependent(&mut self, written: &CppWritten) -> Res { + let Some(node) = self.node_by_id(&written.site.from_node_id)? else { return Ok(false) }; + let parameters = self.cpp_template_parameter_names(&node)?; + Ok(!parameters.is_empty() && type_depends_on(&written.text, ¶meters, 0)) + } + + /// Whether the type `raw`, which nothing the file sees declares, is written under a namespace + /// the project declares nothing in (`absl::Status`, `grpc::ServerContext`), so that it is from + /// outside the project. Not when its first name is a template parameter, a namespace alias or a + /// namespace a macro opens. A lowercase name, and anything of `std`, is the library type check's: + /// a trait or a member type may name one of the project's. + pub(super) fn cpp_outside_qualified_type(&mut self, raw: &str, site: &ResolveRefIn) -> Res { + let Some(parts) = type_segments(&cpp_plain_type(raw)) else { return Ok(false) }; + if parts.len() < 2 { return Ok(false); } + let head = &parts[0]; + if head == "std" || !parts[parts.len() - 1].starts_with(|c: char| c.is_ascii_uppercase()) { return Ok(false); } + if let Some(node) = self.node_by_id(&site.from_node_id)? { + if self.cpp_template_parameter_names(&node)?.contains(head) { return Ok(false); } + } + if self.namespace_frames(&site.file_path)?.iter().any(|(_, _, path)| path.contains(head)) { return Ok(false); } + Ok(!self.cpp_project_scope_names()?.contains(head)) + } + + /// The type of the initializer `value` of an `auto` local, as written where it is declared, with + /// its aliases followed. None when it can't be read reliably, or what a template parameter + /// decides: protobuf's `DownCastMessage` returns a `const T&`, a `T*` or a + /// `std::shared_ptr`, by its argument. + pub(super) fn cpp_initializer_written(&mut self, value: TsNode, source: &Rc, r: &ResolveRefIn) -> Res> { + let Some(written) = self.cpp_expression_written(value, source, r, 0)? else { return Ok(None) }; + let Some(followed) = self.cpp_follow_written_aliases(written, r)? else { return Ok(None) }; + if self.cpp_written_template_dependent(&followed)? { return Ok(None); } + Ok(Some(followed)) + } + + /// A chained call `inner().method` whose inner call's return type is read as its declaration + /// writes it: `method` is that class's, looked up as C++ looks the type up there, or no project + /// method's. By the type's last name, protobuf's `factory->NewEnumGenerator(…)->Generate(…)` in + /// the Java generator reached `csharp::EnumGenerator`. + /// - `Some(Some(_))`: the method of that class or of a class it derives from. + /// - `Some(None)`: the type is from outside the project, or a class that surely lacks the method. + /// - `None`: the type can't be read or placed; the last-name path decides, as before. + pub(super) fn cpp_chain_on_written_type(&mut self, r: &ResolveRefIn) -> Res>> { + let Some(chain) = call_chain_re().captures(&r.reference_name).map(|m| (m[1].to_string(), m[2].to_string())) else { return Ok(None) }; + let Some(source) = self.read_file(&r.file_path) else { return Ok(None) }; + let Some(tree) = self.parsed_tree(&source, r) else { return Ok(None) }; + // The call expression the reference names: the one calling `method` on a call spelled `inner()`. + let wanted: String = format!("{}()", chain.0).split_whitespace().collect(); + let mut at = descendant_for_position(tree.root_node(), source.text(), ((r.line - 1).max(0) as usize, r.column.max(0) as usize)); + let (receiver, arrow) = loop { + if at.kind() == "call_expression" { + let function = at.child_by_field_name("function").filter(|f| f.kind() == "field_expression"); + let field = function.and_then(|f| f.child_by_field_name("field")); + if let (Some(function), Some(field)) = (function, field) { + if source.text()[field.byte_range()] == chain.1 { + let mut receiver = function.child_by_field_name("argument"); + while let Some(inner) = receiver.filter(|n| n.kind() == "parenthesized_expression") { + receiver = match named_children(inner).as_slice() { [only] => Some(*only), _ => None }; + } + if let Some(receiver) = receiver.filter(|n| n.kind() == "call_expression" && chain_spelling(&source.text()[n.byte_range()]) == wanted) { + let arrow = function.child_by_field_name("operator").is_some_and(|op| &source.text()[op.byte_range()] == "->"); + break (receiver, arrow); + } + } + } + } + match at.parent() { Some(parent) => at = parent, None => return Ok(None) } + }; + let Some(written) = self.cpp_expression_written(receiver, &source, r, 1)? else { return Ok(None) }; + let Some(followed) = self.cpp_follow_written_aliases(written, r)? else { return Ok(None) }; + if self.cpp_written_template_dependent(&followed)? { return Ok(None); } + // `->` on a pointer or a smart pointer reaches what it points at; `.` on a smart pointer or + // an optional is its own member (`optional.value()`), no method of what it holds. The chain + // is spelled with `.` for either, so the operator is the AST's. + let object = match cpp_pointee(&followed.text).filter(|_| arrow) { Some(text) => CppWritten { text, site: followed.site }, None => followed }; + let plain = cpp_plain_type(&object.text); + if let Some(parts) = type_segments(&plain) { + let last = parts.last().map(String::as_str).unwrap_or(""); + if parts[0] == "std" || (parts[0] == "absl" && last.starts_with(|c: char| c.is_ascii_lowercase())) { return Ok(Some(None)); } + } + let Some(cls) = self.cpp_written_class(&object, r)? else { + return Ok(if self.cpp_outside_qualified_type(&object.text, &object.site)? { Some(None) } else { None }); + }; + if let Some(method) = self.cpp_methods_named(&cls, &chain.1, r)?.into_iter().next() { + return Ok(Some(Some(KCand { node: method, confidence: 0.85, resolved_by: "instance-method" }))); + } + if self.supertypes_complete && !self.cpp_alias_template_owner(&cls, r)? && self.cpp_bases_known(&cls, 0, &mut HashSet::new())? { return Ok(Some(None)); } + Ok(None) + } + /// The type of a C++ expression in `r`'s caller, as written where it is declared. None for /// anything else, or a name declared only outside the caller and its class. fn cpp_expression_written(&mut self, node: TsNode, source: &Rc, r: &ResolveRefIn, depth: u32) -> Res> { @@ -329,8 +518,33 @@ impl KernelResolver { let Some((cls, field)) = self.cpp_member_of_expression(node, source, r, depth)? else { return Ok(None) }; self.cpp_data_member_written(&cls.qualified_name, &field, r) } + // `new Foo(…)`: a pointer to what it constructs. + "new_expression" => { + let Some(ty) = node.child_by_field_name("type") else { return Ok(None) }; + let text = source.text()[ty.byte_range()].split_whitespace().collect::>().join(" "); + Ok(Some(CppWritten { text: format!("{text}*"), site: r.clone() })) + } + // `(Foo*)arg`: the type it casts to. + "cast_expression" => { + let Some(ty) = node.child_by_field_name("type") else { return Ok(None) }; + let text = source.text()[ty.byte_range()].split_whitespace().collect::>().join(" "); + Ok(Some(CppWritten { text, site: r.clone() })) + } "call_expression" => { let Some(function) = node.child_by_field_name("function") else { return Ok(None) }; + let callee = source.text()[function.byte_range()].trim(); + // `static_cast(…)`: the type it casts to. + if let Some(cast) = re!(r"^(?:static|dynamic|const|reinterpret)_cast\s*<([\s\S]+)>$").captures(callee) { + return Ok(Some(CppWritten { text: cast[1].trim().to_string(), site: r.clone() })); + } + // `std::make_unique(…)`: the smart pointer it makes. + if let Some(made) = re!(r"^(?:::\s*)?(?:std|absl)\s*::\s*make_(unique|shared)\s*<([\s\S]+)>$").captures(callee) { + return Ok(Some(CppWritten { text: format!("std::{}_ptr<{}>", &made[1], made[2].trim()), site: r.clone() })); + } + // `std::unique_ptr(…)`: the smart pointer or optional it makes. + if re!(r"^(?:::\s*)?std\s*::\s*(?:unique_ptr|shared_ptr|optional)\s*<[\s\S]+>$").is_match(callee) { + return Ok(Some(CppWritten { text: callee.to_string(), site: r.clone() })); + } match function.kind() { // `recv.method(…)`, `recv->method(…)`: the method of the receiver's class. "field_expression" => { @@ -338,11 +552,21 @@ impl KernelResolver { let methods = self.cpp_methods_named(&cls, &method, r)?; self.cpp_return_type_of(&methods, r) } - // `f(…)`, `ns::f(…)`, `Class::f(…)`: looked up from the caller's scope outwards. - "identifier" | "qualified_identifier" => { - let spelled: String = source.text()[function.byte_range()].split_whitespace().collect(); + // `f(…)`, `ns::f(…)`, `Class::f(…)`, `Foo(…)`: the caller's class's own or + // inherited method first, then the function looked up from the caller's scope + // outwards, then the class it constructs. + "identifier" | "qualified_identifier" | "template_function" => { + let spelled: String = strip_template_arguments(callee).split_whitespace().collect(); if !re!(r"^(?:[A-Za-z_]\w*::)*[A-Za-z_]\w*$").is_match(&spelled) { return Ok(None); } let Some(caller) = self.node_by_id(&r.from_node_id)? else { return Ok(None) }; + if !spelled.contains("::") { + if let Some(owner) = self.cpp_member_owner(&caller)? { + if let Some(cls) = self.cpp_class_decls(&owner)?.into_iter().next() { + let own = self.cpp_methods_named(&cls, &spelled, r)?; + if !own.is_empty() { return self.cpp_return_type_of(&own, r); } + } + } + } let scopes: Vec = caller.qualified_name.split("::").map(str::to_string).collect(); for i in (0..scopes.len()).rev() { let qualified = if i == 0 { spelled.clone() } else { format!("{}::{spelled}", scopes[..i].join("::")) }; @@ -350,7 +574,9 @@ impl KernelResolver { .filter(|n| matches!(n.kind.as_str(), "method" | "function") && matches!(n.language.as_str(), "c" | "cpp")).cloned().collect(); if !functions.is_empty() { return self.cpp_return_type_of(&functions, r); } } - Ok(None) + // `Foo(…)`, `Foo(…)`: what it constructs, when C++ lookup finds the class. + let constructed = CppWritten { text: callee.split_whitespace().collect::>().join(" "), site: r.clone() }; + Ok(self.cpp_written_class(&constructed, r)?.map(|_| constructed)) } _ => Ok(None), } @@ -388,8 +614,10 @@ impl KernelResolver { if raw.is_empty() || raw.contains('[') { return Ok(None); } if !is_cpp_auto(&raw) { return Ok(Some(CppWritten { text: raw, site: r.clone() })); } // `const auto& blob_files = vstorage->GetBlobFiles();`: what its initializer is. + // (Read with a depth budget of its own: a chain of calls on the local, whose own + // initializer is a chain, gets no shorter for how it was reached.) let Some(value) = binding_initializer(parent, at, name, source) else { return Ok(None) }; - return self.cpp_expression_written(value, source, r, depth + 1); + return self.cpp_expression_written(value, source, r, 0); } if parent.kind() == "for_range_loop" && parent.child_by_field_name("body").is_some_and(|body| body.id() == at.id()) && parent.child_by_field_name("declarator").is_some_and(|d| cpp_declarator_name(d, source) == name) { @@ -402,9 +630,7 @@ impl KernelResolver { } // Declared nowhere in the calling function (`versions_` in db_impl_write.cc, whose class is // declared in db_impl.h): a data member of its class. - let Some(caller) = self.node_by_id(&r.from_node_id)? else { return Ok(None) }; - let Some(owner) = self.cpp_member_owner(&caller)? else { return Ok(None) }; - self.cpp_data_member_written(&owner, name, r) + self.cpp_caller_member(name, r) } /// The element type of a range of the type `range`, as written: a container's first template diff --git a/codegraph-kernel/src/resolve/cpp_receivers.rs b/codegraph-kernel/src/resolve/cpp_receivers.rs index 0ce4bba2b5..fc6fe5f583 100644 --- a/codegraph-kernel/src/resolve/cpp_receivers.rs +++ b/codegraph-kernel/src/resolve/cpp_receivers.rs @@ -6,6 +6,8 @@ impl KernelResolver { /// Plain receivers use lexical AST declarations before any backward text scan. pub(super) fn cpp_plain_call(&mut self, r: &ResolveRefIn) -> Res>> { if r.language != "cpp" || r.reference_kind != "calls" { return Ok(None); } + // `a->b()->c()`: the class `b` is declared to return, not a namesake of it. + if r.reference_name.contains("()") { return self.cpp_chain_on_written_type(r); } let method = r.reference_name.rsplit(['.', '>']).next().unwrap_or(&r.reference_name); if !re!(r"^[A-Za-z_]\w*$").is_match(method) { return Ok(None); } let Some(source) = self.read_file(&r.file_path) else { return Ok(None) }; @@ -32,8 +34,14 @@ impl KernelResolver { let mut site = r.clone(); let mut element = false; let mut scope = receiver; + // The initializer of an `auto` local, which gives it its type. + let mut initializer = None; while let Some(parent) = scope.parent() { - if let Some(ty) = cpp_binding_type(parent, scope, name, &source) { raw = Some(ty); break; } + if let Some(ty) = cpp_binding_type(parent, scope, name, &source) { + if super::cpp_range_for::is_cpp_auto(&ty) { initializer = super::cpp_range_for::binding_initializer(parent, scope, name, &source); } + raw = Some(ty); + break; + } if parent.kind() == "for_range_loop" && parent.child_by_field_name("declarator").is_some_and(|n| cpp_declarator_name(n, &source) == name) { raw = parent.child_by_field_name("type").zip(parent.child_by_field_name("declarator")).map(|(ty, declarator)| cpp_ast_declared_type(ty, declarator, &source)); // A loop over `auto` elements has the element type of its range. @@ -58,6 +66,24 @@ impl KernelResolver { } } } + // A member the class gets from a class it derives from, or a gtest fixture's: where the + // function lacks the name, C++ looks in its class and the classes it derives from. + if raw.is_none() { + if let Some(member) = self.cpp_caller_member(name, r)? { + raw = Some(member.text); + site = member.site; + } + } + // An `auto` local has the type its initializer has where that is written: the cast or smart + // pointer, the class it constructs, or what the function or method called is declared to + // return. The index keeps only a return type's last name, which a namesake shares. + if raw.as_deref().is_some_and(super::cpp_range_for::is_cpp_auto) && !element { + let Some(value) = initializer else { return Ok(None) }; + let Some(written) = self.cpp_initializer_written(value, &source, r)? else { return Ok(None) }; + raw = Some(super::cpp_range_for::cpp_plain_type(&written.text)); + site = written.site; + element = true; + } let Some(raw) = raw else { return Ok(None) }; if super::cpp_range_for::is_cpp_auto(&raw) { return Ok(None); } if raw.contains('[') { return Ok(Some(None)); } @@ -278,9 +304,15 @@ impl KernelResolver { } pub(super) fn cpp_field_type(&mut self, owner: &Arc, field: &str, r: &ResolveRefIn) -> Res> { - let Some(source) = self.read_file(&owner.file_path) else { return Ok(None) }; + Ok(match self.cpp_field_types(owner, field, r)?.as_slice() { [only] => Some(only.clone()), _ => None }) + } + + /// Every type `owner`'s own declaration writes for its data member `field`: one for a member the + /// class declares, none for a name it doesn't, several for one it declares more than once. + pub(super) fn cpp_field_types(&mut self, owner: &Arc, field: &str, r: &ResolveRefIn) -> Res> { + let Some(source) = self.read_file(&owner.file_path) else { return Ok(Vec::new()) }; let mut site = r.clone().at(owner); site.from_node_id = owner.id.clone(); site.column = owner.start_column; - let Some(tree) = self.cpp_field_tree(&source, &site) else { return Ok(None) }; + let Some(tree) = self.cpp_field_tree(&source, &site) else { return Ok(Vec::new()) }; let mut node = descendant_for_position(tree.root_node(), source.text(), ((owner.start_line - 1).max(0) as usize, owner.start_column.max(0) as usize)); while !matches!(node.kind(), "class_specifier" | "struct_specifier" | "union_specifier") { let Some(parent) = node.parent() else { break }; node = parent; @@ -299,7 +331,7 @@ impl KernelResolver { } } } - Ok(match types.as_slice() { [only] => Some(only.clone()), _ => None }) + Ok(types) } fn cpp_range_binding(&self, node: tree_sitter::Node, source: &SourceFile) -> bool { @@ -374,8 +406,12 @@ pub(super) fn cpp_declarator_name<'a>(mut node: tree_sitter::Node, source: &'a S } pub(super) fn cpp_binding_type(parent: tree_sitter::Node, at: tree_sitter::Node, name: &str, source: &SourceFile) -> Option { - if matches!(parent.kind(), "compound_statement" | "declaration_list" | "translation_unit") { - for declaration in named_children(parent).into_iter().rev().filter(|n| n.kind() == "declaration" && n.end_byte() <= at.start_byte()) { + // A `for` or `if` statement's initializer declares a name for the statement, as a block's declaration does. + let block = matches!(parent.kind(), "compound_statement" | "declaration_list" | "translation_unit"); + let initializer = super::cpp_range_for::statement_initializer(parent); + if block || initializer.is_some() { + let children = if block { named_children(parent) } else { initializer.into_iter().collect() }; + for declaration in children.into_iter().rev().filter(|n| n.kind() == "declaration" && n.end_byte() <= at.start_byte()) { // C++ parses a parenthesized initializer as a function declarator. // Recognize only a standard smart pointer initialized from a known pointer value. if let Some(declarator) = declaration.child_by_field_name("declarator").filter(|n| n.kind() == "function_declarator") { diff --git a/codegraph-kernel/src/resolve/mod.rs b/codegraph-kernel/src/resolve/mod.rs index a6ad805ce6..fd41ce35d8 100644 --- a/codegraph-kernel/src/resolve/mod.rs +++ b/codegraph-kernel/src/resolve/mod.rs @@ -635,6 +635,8 @@ pub struct KernelResolver { c_static_memo: HashMap, /// cppMemberMacroArguments, by class declaration id. cpp_macro_arguments_memo: HashMap>>, + /// cppProjectScopeNames: every name a C or C++ qualifier may start with in this project. + cpp_scope_names: Option>>, rust_trait_memo: HashMap, /// javaSupertypesOf, by type name. java_supers_memo: HashMap>>, @@ -848,6 +850,7 @@ impl KernelResolver { esm_exports_memo: HashMap::new(), c_static_memo: HashMap::new(), cpp_macro_arguments_memo: HashMap::new(), + cpp_scope_names: None, rust_trait_memo: HashMap::new(), java_supers_memo: HashMap::new(), php_supers_memo: HashMap::new(),