fix(cpp): a declaration inside a string literal is not a receiver's declaration

CHANGELOG.md

@@ -98,6 +98,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 - In C++, a call a method makes with no receiver, or through `this->`, now links to the method C++ itself would call: one of the method's own class, else one the class inherits from a base class, else one of a class it is nested in. The same goes for a call in a lambda inside the method, and of a class's overloads the one the call's arguments fit is taken. Before, the call was matched by its name alone, so another class's method of that name could win because it was declared nearer or found first: in protocolbuffers/protobuf, every generated message's `Swap` and move assignment calling `InternalSwap(…)` reached `Any::InternalSwap`, and the `GetArena()` messages inherit from `MessageLite` reached an arena helper's `GetArena`, while googletest's `~linked_ptr() { depart(); }`, vendored in facebook/rocksdb, reached `linked_ptr_internal::depart`. Calls C++ doesn't look up in the class are matched as before: a free function's, a call to a parameter or local of that name, and a template's bare call into a base class that depends on its template parameters. Re-index C++ projects after upgrading. Thanks @danusha2345 for reporting the C++ case in #1952.
 - In C++, a call on the variable of a range-based `for` loop over `auto` elements, like `cfd->Ref()` inside facebook/rocksdb's `for (auto cfd : cfds)`, now links to the method of the type the loop goes over. That type is read from what the loop iterates: a `std::vector`, `std::set`, `absl::Span` or similar container, protocolbuffers/protobuf's `RepeatedPtrField` or RocksDB's `autovector`, whether a variable, parameter or member holds it, an `auto` variable is set to it or a function or method returns it, and a project class you loop over through its `begin()`, like RocksDB's `ColumnFamilySet`. Before, the call was matched by the variable's name, or by an earlier variable of that name: in RocksDB, `blob_file.GetBlobFileNumber()` over a list of `BlobFileGarbage` reached `BlobFileAddition::GetBlobFileNumber`, and in protobuf's Objective-C generator, calls over a list of `std::unique_ptr<MessageGenerator>` reached other generators' methods. A call on an element of a standard-library type, like `file.find(…)` over a `std::vector<std::string>`, no longer links to a project method, and `->` on an element held in a `std::unique_ptr` or `std::shared_ptr` reaches the type it holds. Re-index C++ projects after upgrading.
 - In C++, a call on a variable, parameter or member declared as one of your classes, like `iter->Valid()` on an `Iterator* iter` or `it.Valid()` on a `const Iterator& it`, now links to that method of the class C++ itself would find for the type where the call is written, or of a class it inherits from. Before, the method was picked by the type's last name, so a class of the same name nested elsewhere could win: in facebook/rocksdb, calls on an `Iterator` reached `MemTableRep::Iterator`, which declares `Valid` and `Next` itself while the `Iterator` the code means inherits them, google/leveldb's reached `SkipList::Iterator`, and in protocolbuffers/protobuf the Java generator's `context->options()` reached the hpb generator's `Context` and `text_parser.Parse(…)` on a `TextFormat::Parser` reached the `.proto` parser. When that class and the classes it derives from have no method of that name, the call no longer links to an unrelated class's method that shares it, as protobuf's `field->full_name()` on a `FieldDescriptor*` did. A type is also found through a `using` declaration the file writes, like protobuf's `using google::protobuf::Descriptor;`, before a class of the same name declared at global scope elsewhere in the repository; this also applies to base classes and to the type a smart pointer holds. Re-index C++ projects after upgrading.
+- In C++, text inside a string literal is no longer taken for a variable's declaration. That covers the C code protocolbuffers/protobuf's upb generator writes in a raw string (`R"cc( … upb_MiniTableField field = $field_init$; … )cc"`) and log messages like facebook/rocksdb's `"… in WriteBatch handler; …"`. Before, a later call like `field.ctype()` on the function's own `upb::FieldDefPtr field` parameter, or `handler->PutCF(…)` on a `WriteBatch::Handler* handler`, was matched against the type that text seemed to name, so it linked to nothing or to another class's method. Re-index C++ projects after upgrading.
 - Framework support now switches on when an app's `package.json` sits three or more folders deep, as with an ASP.NET solution's Angular or React app in `src/Web/ClientApp/`, or a React app in a workspace folder like `web/ui/mantine-ui/`. Before, only the project root and the first two folder levels were read, so frameworks such as Angular, React Router, Vue Router, SvelteKit and TanStack Router could go undetected there: an Angular app's routes, its `routerLink` and `router.navigate(…)` navigation and its template's event bindings were missing, so a handler used only from a `(click)` binding had no callers, and a React app's `navigate('/login')` calls didn't reach their route. Re-index projects laid out this way after upgrading.
 - In a repository that mixes languages, a web framework's naming conventions now link only code written in that framework's own languages. Before, Express's also linked Go and Python code: in etcd a Go method's `*zap.Logger` result type linked to the method itself, and in Harbor a Python `self.jaeger.validate()` call linked to another class's `validate`. Links that cross languages on purpose, like React Native and Expo native modules, Swift and Objective-C, Drupal's `routing.yml`, SvelteKit's `$lib/…` imports in `.ts` files and a Razor page's `@model`, are unchanged.
 - In C and C++, a call written from the global scope, like `::leveldb::RepairDB(…)` or `::hpb::CreateMessage(…)`, now links to the function of exactly that name, and one through a namespace alias, like protobuf's `::_pbi::…` calls, to the function in the aliased namespace. Before, these calls were dropped unless the repository also held Swift and Objective-C files, and even then a name spelled out in full, like `::google::protobuf::internal::TSanWrite(…)`, linked to nothing, while `::operator delete(…)` linked to one class's own `operator delete`. A global call never links to a function or method of the same name inside a namespace or class, such as a wrapper `open()` that calls the C library's `::open(…)`, and `::std::…` and `::testing::…` calls stay unlinked unless that library's source is part of your project. Re-index C and C++ projects after upgrading.

__tests__/cpp-string-literal-declarations.test.ts

@@ -0,0 +1,182 @@
+/**
+ * A declaration written inside a C++ string literal is text, not code, so
+ * receiver inference doesn't take it for the receiver's declaration.
+ *
+ * protocolbuffers/protobuf's upb generator emits C code from raw string
+ * templates: `GenerateRepeatedSetters(upb::FieldDefPtr field, …)` writes
+ * `upb_MiniTableField field = $field_init$;` inside an `R"cc( … )cc"`, then
+ * calls `field.ctype()` on its own parameter. Reading back from the call
+ * found the template's line first, took `field` for a `upb_MiniTableField` —
+ * a C struct with no methods — and the call got no edge (before #2493, a
+ * guess by the method's name).
+ *
+ * A raw string can span many lines and hold quotes and comment markers, so
+ * its lines are found by reading the file once, as the compiler does. An
+ * ordinary string's contents are blanked on the line being read.
+ */
+import { describe, it, expect, afterEach } from 'vitest';
+import * as fs from 'fs';
+import * as os from 'os';
+import * as path from 'path';
+import { CodeGraph } from '../src';
+import type { Node } from '../src/types';
+
+const roots: string[] = [];
+afterEach(() => {
+  for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true });
+});
+
+async function indexed(files: Record<string, string>): Promise<CodeGraph> {
+  const root = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cpp-strdecl-'));
+  roots.push(root);
+  for (const [rel, content] of Object.entries(files)) {
+    fs.mkdirSync(path.dirname(path.join(root, rel)), { recursive: true });
+    fs.writeFileSync(path.join(root, rel), content);
+  }
+  return CodeGraph.init(root, { index: true });
+}
+
+function callable(cg: CodeGraph, qualifiedName: string): Node {
+  const node = [...cg.getNodesByKind('method'), ...cg.getNodesByKind('function')].find((n) => n.qualifiedName === qualifiedName);
+  if (!node) throw new Error(`no function or method ${qualifiedName}`);
+  return node;
+}
+
+/** `calls` callees of a function or method, as `qualifiedName`. */
+function calls(cg: CodeGraph, caller: string): string[] {
+  return cg
+    .getCallees(callable(cg, caller).id)
+    .filter((r) => r.edge.kind === 'calls')
+    .map((r) => r.node.qualifiedName)
+    .sort();
+}
+
+const lines = (...l: string[]): string => [...l, ''].join('\n');
+
+/** upb's field wrapper, and the C struct a generated template declares. */
+const UPB = {
+  'upb/reflection/def.hpp': lines(
+    'namespace upb {',
+    'class FieldDefPtr {',
+    ' public:',
+    '  int ctype() const { return 0; }',
+    '  const char* name() const { return ""; }',
+    '};',
+    '}  // namespace upb',
+  ),
+  'upb/mini_table/field.h': lines(
+    'typedef struct upb_MiniTableField {',
+    '  int number;',
+    '} upb_MiniTableField;',
+  ),
+};
+
+describe('a declaration inside a string literal is not the receiver\'s', () => {
+  it('reproduction: upb\'s generator declares `field` in the C code a raw string emits', async () => {
+    const cg = await indexed({
+      ...UPB,
+      'upb_generator/c/generator.cc': lines(
+        '#include "upb/reflection/def.hpp"',
+        '#include "upb/mini_table/field.h"',
+        'namespace upb {',
+        'namespace generator {',
+        'void Emit(const char* text) {}',
+        'void GenerateRepeatedSetters(upb::FieldDefPtr field) {',
+        '  Emit(R"cc(',
+        '        UPB_INLINE void $0_resize(void* msg, size_t size) {',
+        '          upb_MiniTableField field = $1;',
+        '          // "a quote" and ) in the template',
+        '          upb_Message_Resize(msg, &field, size);',
+        '        }',
+        '      )cc");',
+        '  if (field.ctype() == 1) Emit(field.name());',
+        '}',
+        '}  // namespace generator',
+        '}  // namespace upb',
+      ),
+    });
+    try {
+      expect([...new Set(calls(cg, 'upb::generator::GenerateRepeatedSetters'))]).toEqual([
+        'upb::FieldDefPtr::ctype',
+        'upb::FieldDefPtr::name',
+        'upb::generator::Emit',
+      ]);
+    } finally {
+      cg.close();
+    }
+  });
+
+  it('an ordinary string that reads like a declaration', async () => {
+    const cg = await indexed({
+      'src/runner.h': lines(
+        'namespace demo {',
+        'class Runner {',
+        ' public:',
+        '  void Run() {}',
+        '};',
+        'class Stub {',
+        ' public:',
+        '  void Run() {}',
+        '};',
+        'class Printer {',
+        ' public:',
+        '  void Print(const char* text) {}',
+        '};',
+        '}  // namespace demo',
+      ),
+      'src/generator.cc': lines(
+        '#include "src/runner.h"',
+        'namespace demo {',
+        'void Generate(Printer* printer, Runner* runner) {',
+        '  printer->Print("Stub* runner = nullptr;");',
+        '  runner->Run();',
+        '}',
+        '}  // namespace demo',
+      ),
+    });
+    try {
+      expect(calls(cg, 'demo::Generate')).toEqual(['demo::Printer::Print', 'demo::Runner::Run']);
+    } finally {
+      cg.close();
+    }
+  });
+});
+
+describe('what a literal on the line does not hide', () => {
+  it('a declaration after a character literal holding a quote, or after a comment holding `R"(`', async () => {
+    const cg = await indexed({
+      'src/runner.h': lines(
+        'namespace demo {',
+        'class Runner {',
+        ' public:',
+        '  void Run() {}',
+        '};',
+        'class Stub {',
+        ' public:',
+        '  void Run() {}',
+        '};',
+        'inline Runner* MakeRunner() { return nullptr; }',
+        '}  // namespace demo',
+      ),
+      'src/quotes.cc': lines(
+        '#include "src/runner.h"',
+        'namespace demo {',
+        'void Quote(Stub* runner) {',
+        '  char q = \'"\'; int n = 1\'000; Runner* runner = MakeRunner();',
+        '  runner->Run();',
+        '}',
+        'void Comment(Stub* runner) {',
+        '  /* R"( not a raw string */ Runner* runner = MakeRunner();',
+        '  runner->Run();',
+        '}',
+        '}  // namespace demo',
+      ),
+    });
+    try {
+      expect(calls(cg, 'demo::Quote')).toEqual(['demo::MakeRunner', 'demo::Runner::Run']);
+      expect(calls(cg, 'demo::Comment')).toEqual(['demo::MakeRunner', 'demo::Runner::Run']);
+    } finally {
+      cg.close();
+    }
+  });
+});

src/resolution/cpp-raw-strings.ts

@@ -0,0 +1,135 @@
+/**
+ * C and C++ string literals as text, not code, for the scans that read
+ * declarations off source lines (receiver inference in name-matcher.ts).
+ *
+ * A raw string literal — `R"cc( … )cc"`, which protobuf's generators use for
+ * the code they emit — can span many lines and hold anything, declarations
+ * included: upb's generator writes `upb_MiniTableField field = $field_init$;`
+ * inside one, and reading back from a later `field.ctype()` on the function's
+ * own `upb::FieldDefPtr field` parameter found that line first. Its lines are
+ * found by reading the file once, as the compiler does: comments, ordinary
+ * string and character literals, and raw strings with their delimiters.
+ * An ordinary string can't span lines, so its contents are blanked on the
+ * line being read.
+ */
+import type { ResolutionContext } from './types';
+
+/** Column ranges, `[start, end)`, a raw string literal takes on a line (its quotes and delimiters included). */
+type Ranges = Array<[number, number]>;
+
+const RAW_STRING_LINES = new WeakMap<ResolutionContext, Map<string, Map<number, Ranges>>>();
+
+/** Drop the per-file raw-string memo (see ReferenceResolver.clearCaches). */
+export function clearCppRawStringMemos(context: ResolutionContext): void {
+  RAW_STRING_LINES.delete(context);
+}
+
+const isWordChar = (c: string | undefined): boolean => c !== undefined && /[A-Za-z0-9_$]/.test(c);
+
+/**
+ * The 1-based lines of a C or C++ file that raw string literals cover, each
+ * with the columns they take. Empty for a file with none.
+ */
+export function cppRawStringLines(file: string, context: ResolutionContext): Map<number, Ranges> {
+  let memo = RAW_STRING_LINES.get(context);
+  if (!memo) RAW_STRING_LINES.set(context, (memo = new Map()));
+  const hit = memo.get(file);
+  if (hit) return hit;
+  const lines = new Map<number, Ranges>();
+  const text = context.readFile(file);
+  if (text && text.includes('R"')) {
+    let line = 1;
+    let column = 0;
+    const n = text.length;
+    let i = 0;
+    // Mark [from, to) of the text, which may span lines, as raw string contents.
+    const mark = (from: number, fromLine: number, fromColumn: number, to: number): void => {
+      let l = fromLine;
+      let start = fromColumn;
+      let c = fromColumn;
+      for (let k = from; k < to; k++) {
+        if (text[k] === '\n') {
+          if (c > start) (lines.get(l) ?? lines.set(l, []).get(l)!).push([start, c]);
+          l++;
+          start = 0;
+          c = 0;
+        } else if (text[k] !== '\r') c++;
+      }
+      if (c > start) (lines.get(l) ?? lines.set(l, []).get(l)!).push([start, c]);
+    };
+    // Advance over [i, to), keeping line and column.
+    const advance = (to: number): void => {
+      for (; i < to; i++) {
+        if (text[i] === '\n') {
+          line++;
+          column = 0;
+        } else if (text[i] !== '\r') column++;
+      }
+    };
+    while (i < n) {
+      const c = text[i]!;
+      const next = text[i + 1];
+      if (c === '/' && next === '/') {
+        const end = text.indexOf('\n', i);
+        advance(end < 0 ? n : end);
+      } else if (c === '/' && next === '*') {
+        const end = text.indexOf('*/', i + 2);
+        advance(end < 0 ? n : end + 2);
+      } else if (c === '"') {
+        // `R"`, `u8R"`, `uR"`, `UR"`, `LR"`, not part of a longer name.
+        const prefix = /(?:u8|[uUL])?R$/.exec(text.slice(Math.max(0, i - 3), i));
+        const open = prefix && !isWordChar(text[i - prefix[0].length - 1]) ? text.indexOf('(', i + 1) : -1;
+        const delimiter = open > i ? text.slice(i + 1, open) : null;
+        if (delimiter !== null && delimiter.length <= 16 && /^[^\s()\\]*$/.test(delimiter)) {
+          // From the opening quote through the closing one: `"cc( … )cc"`.
+          const close = text.indexOf(`)${delimiter}"`, open + 1);
+          const end = close < 0 ? n : close + delimiter.length + 2;
+          mark(i, line, column, end);
+          advance(end);
+        } else {
+          // An ordinary string: to its closing quote, or the end of the line.
+          let k = i + 1;
+          while (k < n && text[k] !== '"' && text[k] !== '\n') k += text[k] === '\\' ? 2 : 1;
+          advance(Math.min(n, text[k] === '"' ? k + 1 : k));
+        }
+      } else if (c === '\'' && !isWordChar(text[i - 1])) {
+        // A character literal (`'"'`); after a digit it is a digit separator (`1'000`).
+        let k = i + 1;
+        while (k < n && text[k] !== '\'' && text[k] !== '\n') k += text[k] === '\\' ? 2 : 1;
+        advance(Math.min(n, text[k] === '\'' ? k + 1 : k));
+      } else advance(i + 1);
+    }
+  }
+  memo.set(file, lines);
+  return lines;
+}
+
+/**
+ * Line `line` (1-based) of a C or C++ file with the contents of its string
+ * and character literals blanked, raw strings included, columns kept.
+ */
+export function cppLiteralsBlanked(text: string, line: number, rawLines: Map<number, Ranges>): string {
+  const raw = rawLines.get(line);
+  const out = text.split('');
+  if (raw) for (const [start, end] of raw) for (let k = start; k < end && k < out.length; k++) out[k] = ' ';
+  if (!text.includes('"') && !text.includes('\'')) return out.join('');
+  const inRaw = (k: number): boolean => !!raw?.some(([start, end]) => k >= start && k < end);
+  for (let k = 0; k < text.length; k++) {
+    const c = text[k]!;
+    if (inRaw(k)) continue;
+    if (c === '/' && text[k + 1] === '/') break;
+    if (c === '/' && text[k + 1] === '*') {
+      // A comment's quotes are no string's (cppCodeOf blanks the comment).
+      const close = text.indexOf('*/', k + 2);
+      if (close < 0) break;
+      k = close + 1;
+      continue;
+    }
+    if (c !== '"' && !(c === '\'' && !isWordChar(text[k - 1]))) continue;
+    let end = k + 1;
+    while (end < text.length && text[end] !== c) end += text[end] === '\\' ? 2 : 1;
+    for (let b = k + 1; b < Math.min(end, text.length); b++) out[b] = ' ';
+    k = end;
+  }
+  return out.join('');
+}