diff --git a/CHANGELOG.md b/CHANGELOG.md index ccd5890c1a..325e1fd962 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -92,7 +92,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). - In C++, a base class like `Message` in `class DynamicMessage final : public Message` now links to the class C++ itself would pick: the one declared in the namespaces around the class, or brought in by the file's `using` declarations and namespace aliases, and the class a `typedef` or `using` alias names. Before, a base class was matched by its name alone, so it could land on a class of the same name in another namespace or another language: in protocolbuffers/protobuf, `DynamicMessage` derived from the JSON parser's internal `Message` class, and in google/leveldb and RocksDB, iterators derived from an unrelated nested `Iterator` class such as the skip list's. The wrong base showed up in `codegraph_explore`'s type hierarchy and in impact, and tied the base's methods to the wrong overrides. A template parameter used as a base no longer links to a class that shares its name, and a base written from the global scope, like `public ::testing::Test`, now links to its class. Re-index C++ projects after upgrading. - In C and C++, a header or source file named like a test, such as protocolbuffers/protobuf's `conformance_test.h` and `test_runner.h`, now counts as part of the code that `#include`s it, directly or through other headers. Before, calls from that code into it were dropped, so protobuf's conformance suites' `suite_.ReportFailure(…)` and `RunValidInputTest(…)` calls, and its unit tests' calls to the shared `TestUtil` helpers, linked to nothing. A test file that only tests include is still kept out of reach of the rest of your code. Re-index C and C++ projects after upgrading. - In C++, a class, struct or method now keeps the namespaces and the class it is declared in when the parser misreads something earlier in the file, such as an unknown macro in front of a member or in a class header. Before, everything after the misread code could lose them, or land inside a class it isn't in: in protocolbuffers/protobuf, `FieldDescriptor` was indexed without `google::protobuf::`, and in RocksDB, the `Opts` struct declared inside a cache table class was indexed outside it, so finding a class's base classes and the methods called on its objects fell back to guessing by name. Methods that had come loose from their class are its members again, and classes the misread code had hidden, like leveldb's POSIX file and environment classes, are now indexed. Re-index C++ projects after upgrading. -- In C++, a call on a variable, parameter or member declared as a standard-library or other outside type with a lowercase name, like `std::string`, `std::vector`, `std::set` or `absl::flat_hash_set`, no longer links to a project method that only shares the method's name. In google/leveldb, `saved_key_.clear()` on a `std::string saved_key_;` reached `Slice::clear`, and in protocolbuffers/protobuf, `proto.append(…)` on a `std::string` reached an unrelated `LeftoverBuffer::append`. The same goes for `->` through a raw pointer like `std::string* out`, while `->` through a smart pointer, iterator or optional still links to the type it holds. Comments also no longer pass for a variable's declaration, so more calls now link to the method of the type the variable is really declared with. Re-index C++ projects after upgrading. +- In C++, a call on a variable, parameter or member declared as a standard-library or other outside type with a lowercase name, like `std::string`, `std::vector`, `std::set` or `absl::flat_hash_set`, no longer links to a project method that only shares the method's name, even one of a project class in another namespace with the type's name, like your own `util::flat_hash_set`. In google/leveldb, `saved_key_.clear()` on a `std::string saved_key_;` reached `Slice::clear`, and in protocolbuffers/protobuf, `proto.append(…)` on a `std::string` reached an unrelated `LeftoverBuffer::append`. The same goes for `->` through a raw pointer like `std::string* out`, while `->` through a smart pointer, iterator or optional still links to the type it holds. Comments also no longer pass for a variable's declaration, so more calls now link to the method of the type the variable is really declared with. Re-index C++ projects after upgrading. - In C++, a class written with attribute macros the parser can't read now keeps its members. That covers several macros between `class` and the class name, like protocolbuffers/protobuf's generated `class PROTOBUF_EXPORT PROTOBUF_FUTURE_ADD_EARLY_WARN_UNUSED Any final`, a macro with arguments there, like RocksDB's `struct ALIGN_AS(64U) HandleImpl`, or one before a template specialization, and macros on a member: in front of it (`PROTOBUF_FUTURE_ADD_EARLY_NODISCARD`), between a pointer and its name (`PROTOBUF_NONNULL`), after its parameters or its name (`ABSL_ATTRIBUTE_LIFETIME_BOUND`, google/leveldb's `LOCKS_EXCLUDED(mu_)` and `GUARDED_BY(mu_)`), or alone on a line above a comment ({fmt}'s `FMT_BEGIN_EXPORT`). Before, protobuf's generated message classes were missing and their members, like `GetDescriptor`, `GetReflection` and `unknown_fields`, were indexed as functions outside any class, and leveldb's annotated methods were indexed under the annotation's name, so calls to them linked to nothing. Re-index C++ projects after upgrading. - In C++, a call on a range-based `for` loop's variable, like `suite->SetVerbose(verbose)` inside `for (ConformanceTestSuite *suite : suites)`, now links to the method of the type the loop declares. Before, the loop's declaration was never read, so the call was matched by the variable's name alone: in RocksDB, `log->file()` on a `log::Writer` went to `BlobLogWriter::file`, and a loop variable named like a member of its class was taken for that member. The variable counts only inside its loop. Re-index C++ projects after upgrading. - In C++, a call made with `->` on a variable, parameter or member declared as a `std::unique_ptr`, `std::shared_ptr` or `std::optional`, like `iter->Valid()` on a `std::unique_ptr iter`, now links to that method of the type it holds, or of a class that type inherits from, with the type looked up the way C++ does it, so a class of the same name nested elsewhere is not mistaken for it. Before, the call went to whichever project method shared its name: in facebook/rocksdb, `iter->Valid()` reached `ArenaWrappedDBIter::Valid` and `fs->GetFileSize(…)` on a `std::shared_ptr` reached a test's stand-in file system, and calls like `it->Next()` were often not linked at all. When what it holds is a template parameter or a standard-library type, like `std::unique_ptr>`, or a class that has no method of that name and neither do the classes it derives from, the call no longer links to a project method; a class derived from something that can't be followed, like a `decltype(…)`, may still have it, so that call is matched as before. Re-index C++ projects after upgrading. @@ -100,7 +100,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). - In C++, a call on the variable of a range-based `for` loop over `auto` elements, like `cfd->Ref()` inside facebook/rocksdb's `for (auto cfd : cfds)`, now links to the method of the type the loop goes over. That type is read from what the loop iterates: a `std::vector`, `std::set`, `absl::Span` or similar container, protocolbuffers/protobuf's `RepeatedPtrField` or RocksDB's `autovector`, whether a variable, parameter or member holds it, an `auto` variable is set to it or a function or method returns it, and a project class you loop over through its `begin()`, like RocksDB's `ColumnFamilySet`. Before, the call was matched by the variable's name, or by an earlier variable of that name: in RocksDB, `blob_file.GetBlobFileNumber()` over a list of `BlobFileGarbage` reached `BlobFileAddition::GetBlobFileNumber`, and in protobuf's Objective-C generator, calls over a list of `std::unique_ptr` reached other generators' methods. A call on an element of a standard-library type, like `file.find(…)` over a `std::vector`, no longer links to a project method, and `->` on an element held in a `std::unique_ptr` or `std::shared_ptr` reaches the type it holds. Re-index C++ projects after upgrading. - In C++, a call on a variable, parameter or member declared as one of your classes, like `iter->Valid()` on an `Iterator* iter` or `it.Valid()` on a `const Iterator& it`, now links to that method of the class C++ itself would find for the type where the call is written, or of a class it inherits from. Before, the method was picked by the type's last name, so a class of the same name nested elsewhere could win: in facebook/rocksdb, calls on an `Iterator` reached `MemTableRep::Iterator`, which declares `Valid` and `Next` itself while the `Iterator` the code means inherits them, google/leveldb's reached `SkipList::Iterator`, and in protocolbuffers/protobuf the Java generator's `context->options()` reached the hpb generator's `Context` and `text_parser.Parse(…)` on a `TextFormat::Parser` reached the `.proto` parser. When that class and the classes it derives from have no method of that name, the call no longer links to an unrelated class's method that shares it, as protobuf's `field->full_name()` on a `FieldDescriptor*` did. A type is also found through a `using` declaration the file writes, like protobuf's `using google::protobuf::Descriptor;`, before a class of the same name declared at global scope elsewhere in the repository; this also applies to base classes and to the type a smart pointer holds. Re-index C++ projects after upgrading. - In C++, text inside a string literal is no longer taken for a variable's declaration. That covers the C code protocolbuffers/protobuf's upb generator writes in a raw string (`R"cc( … upb_MiniTableField field = $field_init$; … )cc"`) and log messages like facebook/rocksdb's `"… in WriteBatch handler; …"`. Before, a later call like `field.ctype()` on the function's own `upb::FieldDefPtr field` parameter, or `handler->PutCF(…)` on a `WriteBatch::Handler* handler`, was matched against the type that text seemed to name, so it linked to nothing or to another class's method. Re-index C++ projects after upgrading. -- In C++, a call on an `auto` variable now links to the method of the class C++ finds for the type its initializer gives it, as a call on a variable declared with that type does. The type is read from a cast or smart pointer written out, like facebook/rocksdb's `auto iter = std::unique_ptr(…)` or `auto* mutex = static_cast(arg)`, from `new` or `std::make_unique` of a class, and from what a function or method is declared to return, like the `Iterator*` of `txn->GetIterator(…)`, read to the end of an initializer that goes on to the next line. Before, such a variable took the type of an earlier variable of the same name, or none: in RocksDB, `iter->SeekToFirst()` reached `MemTableRep::Iterator` and `mutex->Lock()` reached `port::Mutex`. A chained call like `cfd->current()->storage_info()` now looks up the class the inner call is declared to return the same way, through a smart pointer too, so protocolbuffers/protobuf's Java generator no longer reaches the C# generator's `EnumGenerator`. A type from outside the project no longer links to a class of the project that shares its name: protobuf's `status.ok()` on an `absl::Status` reached `upb::Status::ok`, and fmtlib/fmt's `os.flush()` on a `std::ostream&` reached fmt's own `ostream::flush`. Re-index C++ projects after upgrading. +- In C++, a call on an `auto` variable now links to the method of the class C++ finds for the type its initializer gives it, as a call on a variable declared with that type does. The type is read from a cast or smart pointer written out, like facebook/rocksdb's `auto iter = std::unique_ptr(…)` or `auto* mutex = static_cast(arg)`, from `new` or `std::make_unique` of a class, and from what a function or method is declared to return, like the `Iterator*` of `txn->GetIterator(…)`, read to the end of an initializer that goes on to the next line. Before, such a variable took the type of an earlier variable of the same name, or none: in RocksDB, `iter->SeekToFirst()` reached `MemTableRep::Iterator` and `mutex->Lock()` reached `port::Mutex`. A chained call like `cfd->current()->storage_info()` now looks up the class the inner call is declared to return the same way, through a smart pointer too, so protocolbuffers/protobuf's Java generator no longer reaches the C# generator's `EnumGenerator`. A type from outside the project no longer links to a class of the project that shares its name: protobuf's `status.ok()` on an `absl::Status` reached `upb::Status::ok`, and fmtlib/fmt's `os.flush()` on a `std::ostream&` reached fmt's own `ostream::flush`. The same holds for a variable declared in a macro body, or typed by what a function in another file returns, and for a class the index doesn't hold whose name only another class shares: protobuf's `_status.ok()` in its `RETURN_IF_ERROR` macro reached `upb::Status::ok`, and `range.options()` on its generated `DescriptorProto::ExtensionRange` reached `Descriptor::ExtensionRange::options`. A namespace a macro stands for is the project's under either name: facebook/rocksdb declares its classes in `namespace ROCKSDB_NAMESPACE` with `#define ROCKSDB_NAMESPACE rocksdb`, so `db->Put(…)` on a `rocksdb::DB*` links to RocksDB's own `DB::Put`. Re-index C++ projects after upgrading. - In C++, a call on a data member of the calling method's class, like `versions_->GetColumnFamilySet()` in facebook/rocksdb's `db_impl_write.cc`, now links to the method of the member's type wherever the class is declared. Before, only the call's own file and the header of the same name were read for the member, and RocksDB declares `DBImpl` in `db_impl.h` while its methods are spread over `db_impl_write.cc`, `db_impl_compaction_flush.cc`, `db_impl_open.cc` and more, so such calls were guessed by the member's name. A member declared below an inline method that uses it, one a base class declares, and, in a gtest `TEST_F` or `TEST_P` body, one of the test's fixture, like RocksDB's `db_`, are found the same way, and a member of a class nested in the caller's class is no longer taken for the caller's own. A call on a member of a standard-library type, like `std::atomic shutting_down_`, no longer links to a project method that only shares the method's name. Re-index C++ projects after upgrading. - In C++, a variable, parameter or member whose declaration puts a qualifier or an annotation around its name is now read for its type, so calls on it link to the declared class's methods: a `const` after the `*` or `&`, like facebook/rocksdb's `Iterator* const iter = iter_guard.get();` or google/leveldb's `Env* const env_;`, a type written before its `const`, like `Slice const& key`, a nullability annotation after the `*`, like protocolbuffers/protobuf's `io::EpsCopyOutputStream* PROTOBUF_NONNULL stream`, and a thread-safety annotation after the name, like LevelDB's `MemTable* imm_ GUARDED_BY(mutex_);`. Before, such a declaration was skipped, or the annotation was taken for its type, so the call took the type of an earlier variable of the same name or was guessed by the method's name: RocksDB's `iter->SeekToFirst()` reached the unrelated `ArenaWrappedDBIter`, and LevelDB's `env_->NewWritableFile(…)` reached `InMemoryEnv`. Re-index C++ projects after upgrading. - In C++, a call on a name a structured binding declares now links to the method of the type C++ gives it. That covers a range-based `for` over pairs or a map, like facebook/rocksdb's `for (auto& [cfh, iter] : cfh_iter_pairs_) { iter->Valid(); }`, and a declaration like protocolbuffers/protobuf's `auto [dst_ext, is_new] = Insert(…);`. A call on what a `std::function` or another function wrapper returns, like `create_iterator()->Valid()` on a `std::function`, links the same way, and so does a call on a class member whose type is written on the line above its name. Before, these calls were guessed by the method's name, and calls on a library container declared that way, like a `std::priority_queue` member's `pop()`, reached a project class's method of that name. Re-index C++ projects after upgrading. diff --git a/__tests__/cpp-outside-type-receiver.test.ts b/__tests__/cpp-outside-type-receiver.test.ts new file mode 100644 index 0000000000..15a3ca7e86 --- /dev/null +++ b/__tests__/cpp-outside-type-receiver.test.ts @@ -0,0 +1,432 @@ +/** + * A C++ receiver declared with a namespace-qualified type calls a member of a + * class that qualified name can be, never of a class that only shares its + * last name. Three cases read that wrong: + * + * - A namespace a macro stands for. facebook/rocksdb declares its types in + * `namespace ROCKSDB_NAMESPACE` under `#define ROCKSDB_NAMESPACE rocksdb`, + * so the index names them `ROCKSDB_NAMESPACE::DB`. Code that writes + * `rocksdb::DB* db; db->Put(…)` read as a call on an outside type and got no + * edge, and `ROCKSDB_NAMESPACE::Cache` did the same where a file declares + * `namespace rocksdb` itself. + * - A lowercase name a project type shares. `absl::flat_hash_set seen; + * seen.insert(…)` reached the project's own `util::flat_hash_set::insert`: + * a lowercase outside type was told apart from the project's only when no + * project type had its name at all. + * - A type whose last name finds only namesakes, where the receiver's + * declaration is none the call-site rules read (a macro body's, an `auto` + * variable's from another file's return type) or the type's namespace is + * one the project uses: protocolbuffers/protobuf's generated + * `DescriptorProto::ExtensionRange`, which the index doesn't hold, reached + * `Descriptor::ExtensionRange::options`, and `absl::Status _status` in a + * macro body `upb::Status::ok`. + */ +import { describe, it, expect, afterEach } from 'vitest'; +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import { CodeGraph } from '../src'; +import type { Node } from '../src/types'; + +const roots: string[] = []; +afterEach(() => { + for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true }); +}); + +async function indexed(files: Record): Promise { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cpp-outside-')); + roots.push(root); + for (const [rel, content] of Object.entries(files)) { + fs.mkdirSync(path.dirname(path.join(root, rel)), { recursive: true }); + fs.writeFileSync(path.join(root, rel), content); + } + return CodeGraph.init(root, { index: true }); +} + +function callable(cg: CodeGraph, qualifiedName: string): Node { + const node = [...cg.getNodesByKind('method'), ...cg.getNodesByKind('function')].find((n) => n.qualifiedName === qualifiedName); + if (!node) throw new Error(`no function or method ${qualifiedName}`); + return node; +} + +/** `calls` callees of a function or method, as `qualifiedName`. */ +function calls(cg: CodeGraph, caller: string): string[] { + return cg + .getCallees(callable(cg, caller).id) + .filter((r) => r.edge.kind === 'calls') + .map((r) => r.node.qualifiedName) + .sort(); +} + +const lines = (...l: string[]): string => [...l, ''].join('\n'); + +describe('a namespace a macro stands for is the project\'s', () => { + const ROCKSDB = { + 'include/rocksdb/rocksdb_namespace.h': lines( + '#ifndef ROCKSDB_NAMESPACE', + '#define ROCKSDB_NAMESPACE rocksdb', + '#endif', + ), + 'include/rocksdb/db.h': lines( + '#include "include/rocksdb/rocksdb_namespace.h"', + 'namespace ROCKSDB_NAMESPACE {', + 'class DB {', + ' public:', + ' int Put(int key) { return key; }', + '};', + '} // namespace ROCKSDB_NAMESPACE', + ), + 'upb/status.h': lines( + 'namespace upb {', + 'class Status {', + ' public:', + ' bool ok() const { return true; }', + '};', + '} // namespace upb', + ), + }; + + it('reproduction: `rocksdb::DB* db; db->Put(…)` is rocksdb\'s own DB', async () => { + const cg = await indexed({ + ...ROCKSDB, + 'java/rocksjni/put.cc': lines( + '#include "include/rocksdb/db.h"', + 'int Put(rocksdb::DB* db) {', + ' return db->Put(1);', + '}', + ), + }); + try { + expect(calls(cg, 'Put')).toEqual(['ROCKSDB_NAMESPACE::DB::Put']); + } finally { + cg.close(); + } + }); + + it('under either name, while a namespace the project declares nothing in stays outside', async () => { + // This tree declares its types in `namespace rocksdb` itself, and code + // names them through the macro. + const cg = await indexed({ + 'include/rocksdb/rocksdb_namespace.h': ROCKSDB['include/rocksdb/rocksdb_namespace.h'], + 'upb/status.h': ROCKSDB['upb/status.h'], + 'cache/cache.h': lines( + 'namespace rocksdb {', + 'class Cache {', + ' public:', + ' void Release(int handle) {}', + '};', + '} // namespace rocksdb', + ), + 'db/release.cc': lines( + '#include "include/rocksdb/rocksdb_namespace.h"', + '#include "cache/cache.h"', + '#include "upb/status.h"', + 'void Drop(ROCKSDB_NAMESPACE::Cache* cache) {', + ' cache->Release(1);', + '}', + 'bool Passed(const absl::Status& status) {', + ' return status.ok();', + '}', + ), + }); + try { + expect(calls(cg, 'Drop')).toEqual(['rocksdb::Cache::Release']); + expect(calls(cg, 'Passed')).toEqual([]); + } finally { + cg.close(); + } + }); +}); + +describe('a lowercase type from outside the project beside a namesake of the project\'s', () => { + const UTIL = { + 'util/hash_set.h': lines( + 'namespace util {', + 'template ', + 'class flat_hash_set {', + ' public:', + ' void insert(const T& v) {}', + '};', + '} // namespace util', + ), + }; + + it('reproduction: `absl::flat_hash_set seen; seen.insert(…)` is not util::flat_hash_set\'s', async () => { + const cg = await indexed({ + ...UTIL, + 'db/seen.cc': lines( + '#include "absl/container/flat_hash_set.h"', + '#include "util/hash_set.h"', + 'void Mark(int v) {', + ' absl::flat_hash_set seen;', + ' seen.insert(v);', + '}', + ), + }); + try { + expect(calls(cg, 'Mark')).toEqual([]); + expect(cg.getCallers(callable(cg, 'util::flat_hash_set::insert').id).filter((r) => r.edge.kind === 'calls')).toEqual([]); + } finally { + cg.close(); + } + }); + + it('the project\'s own, written with its namespace, a namespace alias, or one a macro opens', async () => { + const cg = await indexed({ + ...UTIL, + 'include/fmt/format.h': lines( + '#define FMT_BEGIN_NAMESPACE namespace fmt { inline namespace v11 {', + '#define FMT_END_NAMESPACE } }', + 'FMT_BEGIN_NAMESPACE', + 'namespace detail {', + 'template ', + 'class buffer {', + ' public:', + ' void push_back(const T& value) {}', + '};', + '} // namespace detail', + 'FMT_END_NAMESPACE', + ), + 'other/buffer.h': lines( + 'namespace other {', + 'class buffer {', + ' public:', + ' void push_back(int value) {}', + '};', + '} // namespace other', + ), + 'db/keep.cc': lines( + '#include "util/hash_set.h"', + '#include "include/fmt/format.h"', + 'namespace u = util;', + 'void Keep(int v) {', + ' util::flat_hash_set mine;', + ' mine.insert(v);', + '}', + 'void Alias(int v) {', + ' u::flat_hash_set aliased;', + ' aliased.insert(v);', + '}', + 'void Append(fmt::detail::buffer& buf) {', + ' buf.push_back(\'x\');', + '}', + ), + }); + try { + expect(calls(cg, 'Keep')).toEqual(['util::flat_hash_set::insert']); + expect(calls(cg, 'Alias')).toEqual(['util::flat_hash_set::insert']); + expect(calls(cg, 'Append')).toEqual(['detail::buffer::push_back']); + } finally { + cg.close(); + } + }); +}); + +describe('a qualified type whose last name finds only namesakes', () => { + /** protobuf's descriptor classes: `descriptor.pb.h` is too large to index, `descriptor.pb.cc` is not. */ + const DESCRIPTORS = { + 'src/google/protobuf/descriptor.h': lines( + 'namespace google {', + 'namespace protobuf {', + 'class Descriptor {', + ' public:', + ' class ExtensionRange {', + ' public:', + ' const int& options() const { return options_; }', + ' private:', + ' int options_;', + ' };', + '};', + 'namespace util {', + 'class SimpleFieldComparator {', + ' public:', + ' void set_treat_nan_as_equal(bool value) {}', + '};', + 'class DefaultFieldComparator : public SimpleFieldComparator {};', + '} // namespace util', + '} // namespace protobuf', + '} // namespace google', + ), + 'src/google/protobuf/descriptor.pb.cc': lines( + 'namespace google {', + 'namespace protobuf {', + 'class DescriptorProto::_Internal {', + ' public:', + ' using HasBits = int;', + '};', + 'DescriptorProto::DescriptorProto() {}', + 'void DescriptorProto::Clear() {}', + '} // namespace protobuf', + '} // namespace google', + ), + }; + + it('reproduction: a generated `DescriptorProto::ExtensionRange` is not `Descriptor::ExtensionRange`', async () => { + const cg = await indexed({ + ...DESCRIPTORS, + 'src/google/protobuf/compiler/importer_unittest.cc': lines( + '#include "src/google/protobuf/descriptor.h"', + 'namespace google {', + 'namespace protobuf {', + 'namespace compiler {', + 'int Options(const DescriptorProto::ExtensionRange& range) {', + ' return range.options();', + '}', + '} // namespace compiler', + '} // namespace protobuf', + '} // namespace google', + ), + }); + try { + expect(calls(cg, 'google::protobuf::compiler::Options')).toEqual([]); + } finally { + cg.close(); + } + }); + + it('a class whose methods alone are indexed, a base class\'s method, a template parameter\'s member', async () => { + const cg = await indexed({ + ...DESCRIPTORS, + // A namesake the written name can't be, beside the class only its methods show. + 'upb/reflection/descriptor_proto.h': lines( + 'namespace upb {', + 'class DescriptorProto {};', + '} // namespace upb', + ), + 'hpb/iterator.h': lines( + 'namespace hpb {', + 'namespace internal {', + 'struct ScalarIteratorPolicy {', + ' struct Payload {', + ' void AddOffset(int n) {}', + ' };', + '};', + 'template ', + 'class Iterator {', + ' private:', + ' typename PolicyT::Payload it_;', + ' public:', + ' void Next() { it_.AddOffset(1); }', + '};', + '} // namespace internal', + '} // namespace hpb', + ), + 'src/google/protobuf/compiler/reset.cc': lines( + '#include "src/google/protobuf/descriptor.h"', + 'namespace google {', + 'namespace protobuf {', + 'namespace compiler {', + 'void Reset(google::protobuf::DescriptorProto* proto) {', + ' proto->Clear();', + '}', + 'void Compare() {', + ' google::protobuf::util::DefaultFieldComparator comparator;', + ' comparator.set_treat_nan_as_equal(true);', + '}', + '} // namespace compiler', + '} // namespace protobuf', + '} // namespace google', + ), + }); + try { + expect(calls(cg, 'google::protobuf::compiler::Reset')).toEqual(['google::protobuf::DescriptorProto::Clear']); + expect(calls(cg, 'google::protobuf::compiler::Compare')).toEqual(['google::protobuf::util::SimpleFieldComparator::set_treat_nan_as_equal']); + expect(calls(cg, 'hpb::internal::Iterator::Next')).toEqual(['hpb::internal::ScalarIteratorPolicy::Payload::AddOffset']); + } finally { + cg.close(); + } + }); + + it('a nested type a derived class inherits, named through the derived class', async () => { + // Its namespace is a class of the project's, so the written name is never + // judged: `ns::Derived::Node` is `ns::Base::Node`. + const cg = await indexed({ + 'src/tree.h': lines( + 'namespace ns {', + 'struct Base {', + ' struct Node {', + ' void Visit() {}', + ' };', + '};', + 'struct Derived : Base {};', + '} // namespace ns', + ), + 'src/walk.cc': lines( + '#include "src/tree.h"', + 'void Root() {', + ' ns::Derived::Node node;', + '}', + 'void Walk() {', + ' node.Visit();', + '}', + ), + }); + try { + expect(calls(cg, 'Walk')).toEqual(['ns::Base::Node::Visit']); + } finally { + cg.close(); + } + }); + + it('an outside type no project class shares a name with stays past guessing', async () => { + // The declaration is another function's, so only the rule for a type no + // class has decides: a `->` on it is no guess at the one project `Read`. + const cg = await indexed({ + 'src/stream.h': lines( + 'class Stream {', + ' public:', + ' bool Read(int n) { return n > 0; }', + '};', + ), + 'src/client.cc': lines( + '#include "src/stream.h"', + 'void Open() {', + ' grpc::ClientReader reader;', + '}', + 'bool Pump() {', + ' return reader->Read(1);', + '}', + ), + }); + try { + expect(calls(cg, 'Pump')).toEqual([]); + } finally { + cg.close(); + } + }); + + it('a `->` on an outside holder of a class the project shares a name with reaches what it holds', async () => { + const cg = await indexed({ + 'util/optional.h': lines( + 'namespace util {', + 'template ', + 'class Optional {', + ' public:', + ' bool has_value() const { return true; }', + '};', + '} // namespace util', + ), + 'src/widget.h': lines( + 'class Widget {', + ' public:', + ' void Draw() {}', + '};', + ), + 'src/paint.cc': lines( + '#include "util/optional.h"', + '#include "src/widget.h"', + 'void Load() {', + ' folly::Optional widget;', + '}', + 'void Paint() {', + ' widget->Draw();', + '}', + ), + }); + try { + expect(calls(cg, 'Paint')).toEqual(['Widget::Draw']); + } finally { + cg.close(); + } + }); +}); diff --git a/src/resolution/cpp-namespaces.ts b/src/resolution/cpp-namespaces.ts index e7221d43ed..e149bd5005 100644 --- a/src/resolution/cpp-namespaces.ts +++ b/src/resolution/cpp-namespaces.ts @@ -81,6 +81,15 @@ export function cppMacroNamespaceHeads(context: ResolutionContext): string[] { return [...cppNamespaceMacros(context).openers.values()].map((path) => path[0]!); } +/** + * The project's macros that stand for a single name, by the macro's name: + * `#define ROCKSDB_NAMESPACE rocksdb`, so the namespace rocksdb declares as + * `namespace ROCKSDB_NAMESPACE` is `rocksdb` to code that names it. + */ +export function cppNameMacros(context: ResolutionContext): ReadonlyMap { + return cppNamespaceMacros(context).aliases; +} + /** The line ranges of a C / C++ file each namespace macro opens, with the namespace path it opens. */ export function cppMacroNamespaceFrames(file: string, context: ResolutionContext): Array<{ start: number; end: number; path: string[] }> { let memo = CPP_NS_FRAMES.get(context); diff --git a/src/resolution/name-matcher.ts b/src/resolution/name-matcher.ts index d3c99d0122..f0d5ad222b 100644 --- a/src/resolution/name-matcher.ts +++ b/src/resolution/name-matcher.ts @@ -15,7 +15,7 @@ import { dartImportPrefixes, dartLibrarySees, dartPrefixSees, inSameDartLibrary import { isDartLocallyBound } from './dart-local-scope'; import { breakVbTie, isVbMemberInScope, isVbNestedTypeInScope, isVbTypeQualifiedBy, matchVbTypedCall, preferVbProject, sameVbProject } from './vbnet-receivers'; import { cppAliasedClass, cppAliasedTypeName, cppAliasedTypeWritten, cppDeclaresLocalAlias, cppParentScope, cppScopesWithin, cppTemplateArguments, cppTemplateParameters, cppTypeSegments, isCppPointerType, resolveCppAliasedType, stripCppTemplateArguments, type CppAliasedType } from './cpp-type-aliases'; -import { clearCppNamespaceMemos, cppMacroNamespaceFrames, cppMacroNamespaceHeads, cppNamespaceAliases } from './cpp-namespaces'; +import { clearCppNamespaceMemos, cppMacroNamespaceFrames, cppMacroNamespaceHeads, cppNameMacros, cppNamespaceAliases } from './cpp-namespaces'; import { clearCppRawStringMemos, cppLiteralsBlanked, cppNonCodeLines } from './cpp-raw-strings'; import { cppClassWritten, type CppNameSite } from './cpp-supertypes'; import { cppIncludedFile, cppIncluders } from './cpp-includers'; @@ -10181,8 +10181,9 @@ function cppTypesNamed(name: string, context: ResolutionContext): Node[] { * no project type's (`Foo::kMask & key` reads like a declaration of `key`), * and the project declares no type of that name: in `std` itself for a * `std::` name (googletest's `testing::internal::string` is no - * `std::string`), anywhere for any other, since a namespace a macro opens - * (fmt's `FMT_BEGIN_NAMESPACE`) is in no qualified name. An unqualified name + * `std::string`); for any other, none the written name can be + * (cppFindsOnlyNamesakes), so `absl::flat_hash_set` is abseil's beside a + * project's own `util::flat_hash_set`. An unqualified name * is not known to be std's: fmt's `using namespace std;` is inside `namespace * adl { … }`, and its `basic_string_view` is fmt's own. Nor is a type whose * `>`s outnumber its `<`s: the end of a declaration begun on an earlier line @@ -10200,7 +10201,7 @@ function isCppLibraryType(raw: string, context: ResolutionContext): boolean { if (!/^[a-z]/.test(name) || /_t$/.test(name)) return false; if (segments.slice(0, -1).some((s) => cppTypesNamed(s, context).length > 0)) return false; const declared = cppTypesNamed(name, context); - if (segments[0] !== 'std') return declared.length === 0; + if (segments[0] !== 'std') return declared.length === 0 || cppFindsOnlyNamesakes(raw, context); const spelled = segments.join('::'); return !declared.some((n) => n.qualifiedName === spelled || n.qualifiedName.endsWith(`::${spelled}`)); } @@ -10387,6 +10388,58 @@ function isCppOutsideQualifiedType(raw: string, site: CppNameSite, context: Reso return !cppProjectScopeNames(context).has(head); } +/** + * Is every class the last name of a C++ type written with a namespace + * qualifier finds — `absl::Status`, protobuf's generated + * `DescriptorProto::ExtensionRange` — a namesake, none of them a class that + * qualified name can be? That is so when the project has C or C++ types of + * that last name, or owners of a `method` of that name (the index can hold a + * class's methods without the class), and each one's qualified name ends + * unlike the written one, neither the other's tail. C++ name lookup only adds + * scopes in front of what is written, and the index leaves some out of a name + * (one a macro opens, one a parse error lost). A namespace alias stands for + * what it names, and a macro that names a namespace (`ROCKSDB_NAMESPACE`) for + * that namespace. Never so for a name one of the project's types qualifies, + * for an unqualified name, or for the end of a declaration begun on an + * earlier line. + */ +function cppFindsOnlyNamesakes(raw: string, context: ResolutionContext, method?: string): boolean { + if ((raw.match(//g)?.length ?? 0)) return false; + const segments = cppTypeSegments(raw); + if (!segments || segments.length < 2) return false; + if (segments.slice(0, -1).some((s) => cppTypesNamed(s, context).length > 0)) return false; + const name = segments[segments.length - 1]!; + const alias = cppNamespaceAliases(context).get(segments[0]!); + const written = [segments, ...(alias ? [[...alias.split('::'), ...segments.slice(1)]] : [])] + .flatMap((s) => cppNamespaceSpellings(s.join('::'), context)); + const fits = (owner: string): boolean => written.some((w) => owner === w || owner.endsWith(`::${w}`) || w.endsWith(`::${owner}`)); + const owners = cppTypesNamed(name, context).map((n) => n.qualifiedName); + if (method) { + const methods = context.getMethodMatches?.(name, method, 'cpp') ?? context.getNodesByName(method).filter((m) => + m.kind === 'method' && sameLanguageFamily(m.language, 'cpp') && `::${m.qualifiedName}`.endsWith(`::${name}::${method}`)); + for (const m of methods) owners.push(m.qualifiedName.slice(0, m.qualifiedName.length - method.length - 2)); + } + return owners.length > 0 && !owners.some((owner) => cppNamespaceSpellings(owner, context).some(fits)); +} + +/** + * A C++ qualified name as it is spelled, and with a macro that names a + * namespace read as that namespace: `ROCKSDB_NAMESPACE::DB` is `rocksdb::DB`. + */ +function cppNamespaceSpellings(qualified: string, context: ResolutionContext): string[] { + const macros = cppNameMacros(context); + if (macros.size === 0) return [qualified]; + const named = qualified.split('::').map((s) => macros.get(s) ?? s).join('::'); + return named === qualified ? [qualified] : [qualified, named]; +} + +/** Is a C++ type written as a member of a template parameter where `ref` is made (`typename PolicyT::Payload`)? */ +function isCppTemplateMember(raw: string, ref: UnresolvedRef, context: ResolutionContext): boolean { + const head = cppTypeSegments(raw)?.[0]; + const caller = context.getNodeById?.(ref.fromNodeId); + return head !== undefined && !!caller && cppTemplateParameters(caller, context).has(head); +} + const CPP_SCOPE_NAMES = new WeakMap>(); /** @@ -10394,8 +10447,11 @@ const CPP_SCOPE_NAMES = new WeakMap>(); * code declares — a namespace (`google`, `ROCKSDB_NAMESPACE`), a class, * whatever a qualifier may start with from some scope — and in the * namespaces its macros open (`fmt` for `FMT_BEGIN_NAMESPACE`), which no - * qualified name shows. (Not those of functions: protobuf's conversion - * operators read as methods named `google::protobuf::absl::Cord`.) + * qualified name shows. A macro that stands for a name (`#define + * ROCKSDB_NAMESPACE rocksdb`) brings the other: rocksdb's types are + * `ROCKSDB_NAMESPACE::DB` in the index and `rocksdb::DB` to code that names + * them. (Not those of functions: protobuf's conversion operators read as + * methods named `google::protobuf::absl::Cord`.) */ function cppProjectScopeNames(context: ResolutionContext): Set { const hit = CPP_SCOPE_NAMES.get(context); @@ -10407,6 +10463,10 @@ function cppProjectScopeNames(context: ResolutionContext): Set { for (const segment of n.qualifiedName.split('::')) if (segment) names.add(segment); } } + for (const [macro, name] of cppNameMacros(context)) { + if (names.has(macro)) names.add(name); + else if (names.has(name)) names.add(macro); + } CPP_SCOPE_NAMES.set(context, names); return names; } @@ -13301,6 +13361,23 @@ export function matchMethodCall( // protobuf's `absl::StatusOr m; m->value()` is no member of the // project's own `hpb::StatusOr`. The name strategies decide. if (inferredType && isCppArrowThroughOutsideType(objectOrClass!, cppDecl, ref, context)) inferredType = null; + // Below, the type's last name finds a method of any class of that name. + // A capitalized type written with its namespace whose every such class + // is a namesake (cppFindsOnlyNamesakes) calls none of them, and is past + // guessing by the receiver's name too, as isUndeclaredTypeName rules + // where no class shares the name: protobuf's `absl::Status _status` in + // a macro body reached `upb::Status::ok`, and its generated + // `DescriptorProto::ExtensionRange`, which the index doesn't hold, + // reached `Descriptor::ExtensionRange::options`. A `->` on a value calls + // a member of what it holds, which the name strategies decide. (A + // lowercase type is isCppLibraryType's; a member of a template + // parameter, `typename PolicyT::Payload`, is whatever the template + // argument makes it.) + if (inferredType && /^[A-Z]/.test(inferredType) && cppDecl.raw !== undefined && !cppDecl.aliased && + cppFindsOnlyNamesakes(cppDecl.raw, context, methodName!) && !isCppTemplateMember(cppDecl.raw, ref, context)) { + if (cppDecl.pointer || cppMemberOperator(objectOrClass!, ref, context) !== '->') return null; + inferredType = null; + } } // A pytest test's parameter is what its fixture returns: flaskbb's // `cli_runner.invoke(…)` is click's `CliRunner`, not the project's one `invoke`.