fix(cpp): an auto local holding a library container's iterator calls the element's method through ->

CHANGELOG.md

@@ -95,7 +95,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 - In C++, a call on a variable, parameter or member declared as a standard-library or other outside type with a lowercase name, like `std::string`, `std::vector`, `std::set` or `absl::flat_hash_set`, no longer links to a project method that only shares the method's name, even one of a project class in another namespace with the type's name, like your own `util::flat_hash_set`. A variable declared outside the calling function, like another function's or a file's `std::string text`, no longer links its calls to such a class either. In google/leveldb, `saved_key_.clear()` on a `std::string saved_key_;` reached `Slice::clear`, and in protocolbuffers/protobuf, `proto.append(…)` on a `std::string` reached an unrelated `LeftoverBuffer::append`. The same goes for `->` through a raw pointer like `std::string* out`, while `->` through a smart pointer, iterator or optional still links to the type it holds. Comments also no longer pass for a variable's declaration, so more calls now link to the method of the type the variable is really declared with. Re-index C++ projects after upgrading.
 - In C++, a class written with attribute macros the parser can't read now keeps its members. That covers several macros between `class` and the class name, like protocolbuffers/protobuf's generated `class PROTOBUF_EXPORT PROTOBUF_FUTURE_ADD_EARLY_WARN_UNUSED Any final`, a macro with arguments there, like RocksDB's `struct ALIGN_AS(64U) HandleImpl`, or one before a template specialization, and macros on a member: in front of it (`PROTOBUF_FUTURE_ADD_EARLY_NODISCARD`), between a pointer and its name (`PROTOBUF_NONNULL`), after its parameters or its name (`ABSL_ATTRIBUTE_LIFETIME_BOUND`, google/leveldb's `LOCKS_EXCLUDED(mu_)` and `GUARDED_BY(mu_)`), or alone on a line above a comment ({fmt}'s `FMT_BEGIN_EXPORT`). Before, protobuf's generated message classes were missing and their members, like `GetDescriptor`, `GetReflection` and `unknown_fields`, were indexed as functions outside any class, and leveldb's annotated methods were indexed under the annotation's name, so calls to them linked to nothing. Re-index C++ projects after upgrading.
 - In C++, a call on a range-based `for` loop's variable, like `suite->SetVerbose(verbose)` inside `for (ConformanceTestSuite *suite : suites)`, now links to the method of the type the loop declares. Before, the loop's declaration was never read, so the call was matched by the variable's name alone: in RocksDB, `log->file()` on a `log::Writer` went to `BlobLogWriter::file`, and a loop variable named like a member of its class was taken for that member. The variable counts only inside its loop. Re-index C++ projects after upgrading.
-- In C++, a call made with `->` on a variable, parameter or member declared as a `std::unique_ptr`, `std::shared_ptr` or `std::optional`, like `iter->Valid()` on a `std::unique_ptr<Iterator> iter`, now links to that method of the type it holds, or of a class that type inherits from, with the type looked up the way C++ does it, so a class of the same name nested elsewhere is not mistaken for it. Before, the call went to whichever project method shared its name: in facebook/rocksdb, `iter->Valid()` reached `ArenaWrappedDBIter::Valid` and `fs->GetFileSize(…)` on a `std::shared_ptr<FileSystem>` reached a test's stand-in file system, and calls like `it->Next()` were often not linked at all. When what it holds is a template parameter or a standard-library type, like `std::unique_ptr<std::deque<…>>`, or a class that has no method of that name and neither do the classes it derives from, the call no longer links to a project method; a class derived from something that can't be followed, like a `decltype(…)`, may still have it, so that call is matched as before. The same goes for `->` on a container's iterator, like `it->data()` on a `std::deque<Slice>::const_iterator it` or `loc->path_size()` on protocolbuffers/protobuf's `RepeatedPtrField<SourceCodeInfo_Location>::iterator loc`: the call links to that method of the container's element type, and to no project method when the element is a standard-library type like `std::string`. Re-index C++ projects after upgrading.
+- In C++, a call made with `->` on a variable, parameter or member declared as a `std::unique_ptr`, `std::shared_ptr` or `std::optional`, like `iter->Valid()` on a `std::unique_ptr<Iterator> iter`, now links to that method of the type it holds, or of a class that type inherits from, with the type looked up the way C++ does it, so a class of the same name nested elsewhere is not mistaken for it. Before, the call went to whichever project method shared its name: in facebook/rocksdb, `iter->Valid()` reached `ArenaWrappedDBIter::Valid` and `fs->GetFileSize(…)` on a `std::shared_ptr<FileSystem>` reached a test's stand-in file system, and calls like `it->Next()` were often not linked at all. When what it holds is a template parameter or a standard-library type, like `std::unique_ptr<std::deque<…>>`, or a class that has no method of that name and neither do the classes it derives from, the call no longer links to a project method; a class derived from something that can't be followed, like a `decltype(…)`, may still have it, so that call is matched as before. The same goes for `->` on a container's iterator, like `it->data()` on a `std::deque<Slice>::const_iterator it` or `loc->path_size()` on protocolbuffers/protobuf's `RepeatedPtrField<SourceCodeInfo_Location>::iterator loc`: the call links to that method of the container's element type, and to no project method when the element is a standard-library type like `std::string`. That includes an iterator an `auto` variable gets from a standard-library container's `begin()`, `find()` and the like, or from a standard search like `std::lower_bound`, as in facebook/rocksdb's `auto it = logs_.begin(); it->IsSyncing()`. Re-index C++ projects after upgrading.
 - In C++, a call a method makes with no receiver, or through `this->`, now links to the method C++ itself would call: one of the method's own class, else one the class inherits from a base class, else one of a class it is nested in. The same goes for a call in a lambda inside the method, and of a class's overloads the one the call's arguments fit is taken. Before, the call was matched by its name alone, so another class's method of that name could win because it was declared nearer or found first: in protocolbuffers/protobuf, every generated message's `Swap` and move assignment calling `InternalSwap(…)` reached `Any::InternalSwap`, and the `GetArena()` messages inherit from `MessageLite` reached an arena helper's `GetArena`, while googletest's `~linked_ptr() { depart(); }`, vendored in facebook/rocksdb, reached `linked_ptr_internal::depart`. Calls C++ doesn't look up in the class are matched as before: a free function's, a call to a parameter or local of that name, and a template's bare call into a base class that depends on its template parameters. Re-index C++ projects after upgrading. Thanks @danusha2345 for reporting the C++ case in #1952.
 - In C++, a call on the variable of a range-based `for` loop over `auto` elements, like `cfd->Ref()` inside facebook/rocksdb's `for (auto cfd : cfds)`, now links to the method of the type the loop goes over. That type is read from what the loop iterates: a `std::vector`, `std::set`, `absl::Span` or similar container, protocolbuffers/protobuf's `RepeatedPtrField` or RocksDB's `autovector`, whether a variable, parameter or member holds it, an `auto` variable is set to it or a function or method returns it, and a project class you loop over through its `begin()`, like RocksDB's `ColumnFamilySet`. Before, the call was matched by the variable's name, or by an earlier variable of that name: in RocksDB, `blob_file.GetBlobFileNumber()` over a list of `BlobFileGarbage` reached `BlobFileAddition::GetBlobFileNumber`, and in protobuf's Objective-C generator, calls over a list of `std::unique_ptr<MessageGenerator>` reached other generators' methods. A call on an element of a standard-library type, like `file.find(…)` over a `std::vector<std::string>`, no longer links to a project method, and `->` on an element held in a `std::unique_ptr` or `std::shared_ptr` reaches the type it holds. Re-index C++ projects after upgrading.
 - In C++, a call on a variable, parameter or member declared as one of your classes, like `iter->Valid()` on an `Iterator* iter` or `it.Valid()` on a `const Iterator& it`, now links to that method of the class C++ itself would find for the type where the call is written, or of a class it inherits from. Before, the method was picked by the type's last name, so a class of the same name nested elsewhere could win: in facebook/rocksdb, calls on an `Iterator` reached `MemTableRep::Iterator`, which declares `Valid` and `Next` itself while the `Iterator` the code means inherits them, google/leveldb's reached `SkipList::Iterator`, and in protocolbuffers/protobuf the Java generator's `context->options()` reached the hpb generator's `Context` and `text_parser.Parse(…)` on a `TextFormat::Parser` reached the `.proto` parser. When that class and the classes it derives from have no method of that name, the call no longer links to an unrelated class's method that shares it, as protobuf's `field->full_name()` on a `FieldDescriptor*` did. A type is also found through a `using` declaration the file writes, like protobuf's `using google::protobuf::Descriptor;`, before a class of the same name declared at global scope elsewhere in the repository; this also applies to base classes and to the type a smart pointer holds. A type written from the global scope, like `::upb::Arena& arena`, is looked up there, so a namespace's own `upb::Arena` is not taken for it, nor is an alias of that name the calling function declares. A type written as a member of a class template, like `SkipList<Key, Comparator>::Iterator iter` in leveldb's and RocksDB's skip-list tests, is now read whole, so its calls reach the skip list's own iterator rather than the general `Iterator`, and RocksDB's skip-list memtable iterator no longer links its calls on the skip list's iterator to its own methods; a standard container's `iterator` written this way is the library's, never a class of yours named `iterator`. Re-index C++ projects after upgrading.

__tests__/cpp-auto-iterator-arrow.test.ts

@@ -0,0 +1,248 @@
+/**
+ * A `->` call on a C++ `auto` local that a library container's member or a
+ * standard search hands an iterator, like rocksdb's `auto it =
+ * logs_.begin(); it->IsSyncing()`, calls a member of the container's
+ * element, as on an iterator declared with its type. The local had no type,
+ * so the call was matched by the method's name alone.
+ */
+import { describe, it, expect, afterEach } from 'vitest';
+import * as fs from 'fs';
+import * as os from 'os';
+import * as path from 'path';
+import { CodeGraph } from '../src';
+import type { Node } from '../src/types';
+
+const roots: string[] = [];
+afterEach(() => {
+  for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true });
+});
+
+async function indexed(files: Record<string, string>): Promise<CodeGraph> {
+  const root = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cpp-auto-iterator-'));
+  roots.push(root);
+  for (const [rel, content] of Object.entries(files)) {
+    fs.mkdirSync(path.dirname(path.join(root, rel)), { recursive: true });
+    fs.writeFileSync(path.join(root, rel), content);
+  }
+  return CodeGraph.init(root, { index: true });
+}
+
+function callable(cg: CodeGraph, qualifiedName: string): Node {
+  const node = [...cg.getNodesByKind('method'), ...cg.getNodesByKind('function')].find((n) => n.qualifiedName === qualifiedName);
+  if (!node) throw new Error(`no function or method ${qualifiedName}`);
+  return node;
+}
+
+/** `calls` callees of a function or method, as `qualifiedName`. */
+function calls(cg: CodeGraph, caller: string): string[] {
+  return cg
+    .getCallees(callable(cg, caller).id)
+    .filter((r) => r.edge.kind === 'calls')
+    .map((r) => r.node.qualifiedName)
+    .sort();
+}
+
+const lines = (...l: string[]): string => [...l, ''].join('\n');
+
+describe('a `->` call on a C++ `auto` local holding a library container\'s iterator', () => {
+  it('reproduction: rocksdb\'s `auto it = logs_.begin(); it->IsSyncing()`', async () => {
+    const cg = await indexed({
+      'db/log_writer.h': lines(
+        'class Writer {',
+        ' public:',
+        '  bool IsSyncing() const { return false; }',
+        '};',
+      ),
+      'db/db_impl.h': lines(
+        '#include <deque>',
+        'class DBImpl {',
+        ' public:',
+        '  struct LogWriterNumber {',
+        '    bool IsSyncing() const { return true; }',
+        '    void ReleaseWriter() {}',
+        '  };',
+        '  void MarkLogsSynced();',
+        ' private:',
+        '  std::deque<LogWriterNumber> logs_;',
+        '};',
+      ),
+      'db/db_impl.cc': lines(
+        '#include "db/db_impl.h"',
+        '#include "db/log_writer.h"',
+        'void DBImpl::MarkLogsSynced() {',
+        '  auto it = logs_.begin();',
+        '  if (it->IsSyncing()) it->ReleaseWriter();',
+        '}',
+      ),
+    });
+    try {
+      expect(calls(cg, 'DBImpl::MarkLogsSynced')).toEqual(['DBImpl::LogWriterNumber::IsSyncing', 'DBImpl::LogWriterNumber::ReleaseWriter']);
+    } finally {
+      cg.close();
+    }
+  });
+
+  it('from a standard search, and a reverse iterator', async () => {
+    const cg = await indexed({
+      'util/entries.h': lines(
+        'class Entry {',
+        ' public:',
+        '  const char* name() const { return nullptr; }',
+        '};',
+        'class Column {',
+        ' public:',
+        '  const char* name() const { return nullptr; }',
+        '};',
+      ),
+      'util/lookup.cc': lines(
+        '#include <algorithm>',
+        '#include <vector>',
+        '#include "util/entries.h"',
+        'const char* Find(const std::vector<Entry>& entries, const Entry& key) {',
+        '  auto it = std::lower_bound(entries.begin(), entries.end(), key);',
+        '  return it->name();',
+        '}',
+        'const char* Last(std::vector<Entry>& entries) {',
+        '  auto it = entries.rbegin();',
+        '  return it->name();',
+        '}',
+      ),
+    });
+    try {
+      expect(calls(cg, 'Find')).toEqual(['Entry::name']);
+      expect(calls(cg, 'Last')).toEqual(['Entry::name']);
+    } finally {
+      cg.close();
+    }
+  });
+
+  it('the element is looked up where the container\'s type is written', async () => {
+    const cg = await indexed({
+      'db/version_set.h': lines(
+        '#include <vector>',
+        'struct ObsoleteFile {',
+        '  void DeleteFile() {}',
+        '};',
+        'class VersionSet {',
+        ' public:',
+        '  struct ObsoleteFile {',
+        '    void DeleteFile() {}',
+        '  };',
+        '  std::vector<ObsoleteFile> obsolete_files_;',
+        '};',
+      ),
+      'db/purge.cc': lines(
+        '#include "db/version_set.h"',
+        'void Purge(VersionSet* vs) {',
+        '  auto it = vs->obsolete_files_.begin();',
+        '  it->DeleteFile();',
+        '}',
+      ),
+    });
+    try {
+      expect(calls(cg, 'Purge')).toEqual(['VersionSet::ObsoleteFile::DeleteFile']);
+    } finally {
+      cg.close();
+    }
+  });
+
+  it('a smart pointer member\'s type is still looked up where the call is: a pimpl defined in the source file', async () => {
+    const cg = await indexed({
+      'db/database.h': lines(
+        '#include <memory>',
+        'class SimpleDatabase {',
+        ' public:',
+        '  class DescriptorIndex {',
+        '   public:',
+        '    bool AddFile() { return false; }',
+        '  };',
+        '};',
+        'class EncodedDatabase {',
+        ' public:',
+        '  bool Add();',
+        ' private:',
+        '  class DescriptorIndex;',
+        '  std::unique_ptr<DescriptorIndex> index_;',
+        '};',
+      ),
+      'db/database.cc': lines(
+        '#include "db/database.h"',
+        'class EncodedDatabase::DescriptorIndex {',
+        ' public:',
+        '  bool AddFile() { return true; }',
+        '};',
+        'bool EncodedDatabase::Add() {',
+        '  return index_->AddFile();',
+        '}',
+      ),
+    });
+    try {
+      expect(calls(cg, 'EncodedDatabase::Add')).toEqual(['EncodedDatabase::DescriptorIndex::AddFile']);
+    } finally {
+      cg.close();
+    }
+  });
+
+  it('an element of a library type has no project method to call', async () => {
+    const cg = await indexed({
+      'util/library.h': lines(
+        'class ObjectLibrary {',
+        ' public:',
+        '  ObjectLibrary* get() { return this; }',
+        '};',
+      ),
+      'util/registry.cc': lines(
+        '#include <memory>',
+        '#include <vector>',
+        '#include "util/library.h"',
+        'class Registry {',
+        ' public:',
+        '  bool Newest() const {',
+        '    auto iter = libraries_.crbegin();',
+        '    return iter->get() != nullptr;',
+        '  }',
+        ' private:',
+        '  std::vector<std::shared_ptr<ObjectLibrary>> libraries_;',
+        '};',
+      ),
+    });
+    try {
+      expect(calls(cg, 'Registry::Newest')).toEqual([]);
+    } finally {
+      cg.close();
+    }
+  });
+
+  it('a project class named like a library container keeps what its own `begin()` returns', async () => {
+    const cg = await indexed({
+      'util/span.h': lines(
+        'class Foo {',
+        ' public:',
+        '  void Bar() {}',
+        '};',
+        'class Baz {',
+        ' public:',
+        '  void Bar() {}',
+        '};',
+        'namespace util {',
+        'class span {',
+        ' public:',
+        '  Foo* begin() { return nullptr; }',
+        '};',
+        '}  // namespace util',
+      ),
+      'util/use.cc': lines(
+        '#include "util/span.h"',
+        'void Use(util::span s) {',
+        '  auto it = s.begin();',
+        '  it->Bar();',
+        '}',
+      ),
+    });
+    try {
+      expect(calls(cg, 'Use')).toEqual(['Foo::Bar', 'util::span::begin']);
+    } finally {
+      cg.close();
+    }
+  });
+});