fix(c,cpp): a file named like a test is visible to the code that includes it

CHANGELOG.md

@@ -70,6 +70,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 - In C, C++ and Objective-C, a struct or class member declared as a pointer, reference, array or function, like `SharedState* shared;`, `jv elements[];` or `virtual Status Put(…) = 0;`, no longer makes the struct look like it inherits from the member's type. Before, `codegraph_explore` and impact listed those types as base classes, every struct holding a pointer to a type as one of its subclasses, and a method of that struct as an override of the type's method with the same name. Real base classes like `class Derived : public Base`, Objective-C superclasses and Go's embedded fields are unchanged. Re-index C, C++ and Objective-C projects after upgrading.
 - In C++, a call on a variable, parameter or member declared through a `typedef` or `using` alias now links to the method of the type the alias names, with the alias looked up the way C++ does it: in the calling function, its class and the classes that class inherits from, then the namespaces around it. In google/leveldb, `table_.Insert(buf)` on a `Table table_;` declared next to `typedef SkipList<const char*, KeyComparator> Table;` now reaches `SkipList::Insert` instead of the unrelated `HandleTable::Insert`, and `Table::Iterator iter(&table_);` reaches `SkipList::Iterator`'s constructor. A class that only shares the alias's name is no longer taken for it, and when the aliased type has no such method, as with an alias of `std::vector` or of a template parameter, no method is guessed from the receiver's name. Re-index C++ projects after upgrading.
 - In C++, a base class like `Message` in `class DynamicMessage final : public Message` now links to the class C++ itself would pick: the one declared in the namespaces around the class, or brought in by the file's `using` declarations and namespace aliases, and the class a `typedef` or `using` alias names. Before, a base class was matched by its name alone, so it could land on a class of the same name in another namespace or another language: in protocolbuffers/protobuf, `DynamicMessage` derived from the JSON parser's internal `Message` class, and in google/leveldb and RocksDB, iterators derived from an unrelated nested `Iterator` class such as the skip list's. The wrong base showed up in `codegraph_explore`'s type hierarchy and in impact, and tied the base's methods to the wrong overrides. A template parameter used as a base no longer links to a class that shares its name, and a base written from the global scope, like `public ::testing::Test`, now links to its class. Re-index C++ projects after upgrading.
+- In C and C++, a header or source file named like a test, such as protocolbuffers/protobuf's `conformance_test.h` and `test_runner.h`, now counts as part of the code that `#include`s it, directly or through other headers. Before, calls from that code into it were dropped, so protobuf's conformance suites' `suite_.ReportFailure(…)` and `RunValidInputTest(…)` calls, and its unit tests' calls to the shared `TestUtil` helpers, linked to nothing. A test file that only tests include is still kept out of reach of the rest of your code. Re-index C and C++ projects after upgrading.
 - Framework support now switches on when an app's `package.json` sits three or more folders deep, as with an ASP.NET solution's Angular or React app in `src/Web/ClientApp/`, or a React app in a workspace folder like `web/ui/mantine-ui/`. Before, only the project root and the first two folder levels were read, so frameworks such as Angular, React Router, Vue Router, SvelteKit and TanStack Router could go undetected there: an Angular app's routes, its `routerLink` and `router.navigate(…)` navigation and its template's event bindings were missing, so a handler used only from a `(click)` binding had no callers, and a React app's `navigate('/login')` calls didn't reach their route. Re-index projects laid out this way after upgrading.
 - In Go, an interface that embeds another, like gin's `IRouter` embedding `IRoutes`, now links to it, so `codegraph_explore` and impact show the embedded interface as its base. Before, every embedded interface was dropped. An embedded field written through another package of your project, like `*refresh.Discovery` or `storage.Appender`, or with type arguments, like `List[int]`, now links too. One from the standard library or another module, like `io.Closer` or `sync.Mutex`, links to nothing rather than to a project type that shares its name, and a type-set constraint like `~int | ~float64` is not an embedding. An embedded type written without a package now always links to its own package's type: before, etcd's `Client`, which embeds its own `Lease` interface, was linked to a `Lease` struct in another package. Re-index Go projects after upgrading.
 - In Go, working out which structs implement an interface now counts the methods of the interfaces it embeds, and the methods a struct gets from the types it embeds. Before, only each type's own methods counted: etcd's `AuthReadTx`, which embeds `UnsafeAuthReader` and adds `RLock` and `RUnlock`, was listed as implemented by every read-write lock in the project, an interface that only embeds others had no implementations at all, and a struct that gets its methods from an embedded base, like prometheus's service discoveries embedding `refresh.Discovery` or gin's `Engine` embedding `RouterGroup`, was missing. `codegraph_explore`, impact and the type hierarchy now list the right implementations, and a call through the interface reaches a method the embedding struct overrides, like gin's `Engine.Use`. Re-index Go projects after upgrading.

__tests__/cpp-included-test-suite.test.ts

@@ -0,0 +1,259 @@
+/**
+ * A C or C++ file is part of every translation unit that includes it, whatever
+ * its name says. protobuf's conformance framework lives in
+ * `conformance/conformance_test.h` / `.cc` (`ConformanceTestSuite`) and
+ * `conformance/test_runner.h`, named like test suites, and the suites and
+ * runners that use it include them: `suite_.ReportFailure(…)` in
+ * binary_json_conformance_suite.cc is `ConformanceTestSuite::ReportFailure`.
+ * The test-suite rule ("production code never names a test suite's symbol")
+ * dropped that edge. A test-named file no production file includes stays out
+ * of reach, and so does a test that only includes the header it tests.
+ */
+import { describe, it, expect, afterAll, beforeAll } from 'vitest';
+import * as fs from 'fs';
+import * as os from 'os';
+import * as path from 'path';
+import { CodeGraph } from '../src';
+
+let root = '';
+let cg: CodeGraph;
+
+beforeAll(async () => {
+  root = fs.mkdtempSync(path.join(os.tmpdir(), 'cg-cpp-included-test-'));
+  const files: Record<string, string> = {
+    'conformance/test_runner.h': `#ifndef CONFORMANCE_TEST_RUNNER_H
+#define CONFORMANCE_TEST_RUNNER_H
+
+#include <string>
+
+namespace conformance {
+
+class ConformanceTestRunner {
+ public:
+  virtual ~ConformanceTestRunner() = default;
+  virtual void RunTest(const std::string& test_name, const std::string& input, std::string* output) = 0;
+};
+
+}  // namespace conformance
+
+#endif
+`,
+    'conformance/conformance_test.h': `#ifndef CONFORMANCE_CONFORMANCE_TEST_H
+#define CONFORMANCE_CONFORMANCE_TEST_H
+
+#include <string>
+
+#include "conformance/test_runner.h"
+
+namespace conformance {
+
+class ConformanceTestSuite {
+ public:
+  virtual ~ConformanceTestSuite() = default;
+  bool RunSuite(ConformanceTestRunner* runner, std::string* output);
+  void ReportFailure(int test, int level);
+
+ protected:
+  virtual void RunSuiteImpl() = 0;
+  int failures_ = 0;
+};
+
+int FailureCount(const ConformanceTestSuite& suite);
+
+}  // namespace conformance
+
+#endif
+`,
+    'conformance/conformance_test.cc': `#include "conformance/conformance_test.h"
+
+namespace conformance {
+
+bool ConformanceTestSuite::RunSuite(ConformanceTestRunner* runner, std::string* output) {
+  RunSuiteImpl();
+  return failures_ == 0;
+}
+
+void ConformanceTestSuite::ReportFailure(int test, int level) {
+  failures_ += test + level;
+}
+
+int FailureCount(const ConformanceTestSuite& suite) {
+  return 0;
+}
+
+}  // namespace conformance
+`,
+    // Another class's ReportFailure: where the call went by guess before.
+    'conformance/fork_pipe_runner.h': `#include <string>
+
+#include "conformance/test_runner.h"
+
+namespace conformance {
+
+class ForkPipeRunner : public ConformanceTestRunner {
+ public:
+  void RunTest(const std::string& test_name, const std::string& input, std::string* output) override;
+
+ private:
+  std::string ReportFailure(bool timed_out, const std::string& what_failed);
+};
+
+}  // namespace conformance
+`,
+    'conformance/fork_pipe_runner.cc': `#include "conformance/fork_pipe_runner.h"
+
+namespace conformance {
+
+void ForkPipeRunner::RunTest(const std::string& test_name, const std::string& input, std::string* output) {
+  *output = ReportFailure(false, test_name);
+}
+
+std::string ForkPipeRunner::ReportFailure(bool timed_out, const std::string& what_failed) {
+  return what_failed;
+}
+
+}  // namespace conformance
+`,
+    'conformance/binary_json_conformance_suite.h': `#include "conformance/conformance_test.h"
+
+namespace conformance {
+
+class BinaryAndJsonConformanceSuite : public ConformanceTestSuite {
+ private:
+  void RunSuiteImpl() override;
+};
+
+class BinaryAndJsonConformanceSuiteImpl {
+ public:
+  explicit BinaryAndJsonConformanceSuiteImpl(BinaryAndJsonConformanceSuite* suite);
+  void RunValidBinaryInputTest(int test);
+
+ private:
+  BinaryAndJsonConformanceSuite& suite_;
+};
+
+}  // namespace conformance
+`,
+    'conformance/binary_json_conformance_suite.cc': `#include "conformance/binary_json_conformance_suite.h"
+
+namespace conformance {
+
+void BinaryAndJsonConformanceSuite::RunSuiteImpl() {
+  BinaryAndJsonConformanceSuiteImpl impl(this);
+  impl.RunValidBinaryInputTest(1);
+}
+
+void BinaryAndJsonConformanceSuiteImpl::RunValidBinaryInputTest(int test) {
+  suite_.ReportFailure(test, 0);
+}
+
+}  // namespace conformance
+`,
+    'conformance/conformance_test_main.cc': `#include "conformance/binary_json_conformance_suite.h"
+#include "conformance/conformance_test.h"
+
+int RunConformance(const conformance::BinaryAndJsonConformanceSuite& suite) {
+  return FailureCount(suite);
+}
+`,
+    // Production code that does not include conformance_test.h.
+    'tools/report.cc': `int Summarize(int suite) {
+  return FailureCount(suite);
+}
+`,
+    // A C test-named file a production file includes (redis' expr.c includes
+    // fastjson_test.c for its test build).
+    'modules/fastjson_test.c': `int run_fastjson_test(void) {
+  return 0;
+}
+`,
+    'modules/expr.c': `#include "fastjson_test.c"
+
+int expr_selftest(void) {
+  return run_fastjson_test();
+}
+`,
+    // A real test suite nothing includes: its helper is not production's.
+    'src/db.h': `int open_db(void);
+`,
+    'src/db.c': `#include "db.h"
+
+int open_db(void) {
+  return make_options();
+}
+`,
+    'src/db_test.c': `#include "db.h"
+
+int make_options(void) {
+  return 1;
+}
+`,
+    // A test that includes the header it tests, named like it, does not
+    // implement it: jemalloc's test/unit/hash.c and its hash.h.
+    'include/hash.h': `unsigned hash(const char* key);
+`,
+    'src/hash.c': `#include "hash.h"
+
+unsigned hash(const char* key) {
+  return hash_selftest();
+}
+`,
+    'test/unit/hash.c': `#include "hash.h"
+
+unsigned hash_selftest(void) {
+  return hash("abc");
+}
+`,
+  };
+  for (const [rel, content] of Object.entries(files)) {
+    fs.mkdirSync(path.dirname(path.join(root, rel)), { recursive: true });
+    fs.writeFileSync(path.join(root, rel), content);
+  }
+  cg = await CodeGraph.init(root, { index: true });
+});
+
+afterAll(() => {
+  cg?.close();
+  if (root) fs.rmSync(root, { recursive: true, force: true });
+});
+
+/** `qualifiedName (file)` of every call the file's symbols make. */
+const callsFrom = (file: string): string[] => {
+  const ids = cg.getNodesInFile(file).map((n) => n.id);
+  return cg.getOutgoingEdgesFrom(ids).filter((e) => e.kind === 'calls').map((e) => {
+    const target = cg.getNode(e.target)!;
+    return `${target.qualifiedName} (${target.filePath})`;
+  });
+};
+
+describe('a C/C++ file named like a test that the program includes', () => {
+  it('links a call through the base class its suite derives from', () => {
+    const calls = callsFrom('conformance/binary_json_conformance_suite.cc');
+    expect(calls).toContain('conformance::ConformanceTestSuite::ReportFailure (conformance/conformance_test.cc)');
+    expect(calls).not.toContain('conformance::ForkPipeRunner::ReportFailure (conformance/fork_pipe_runner.cc)');
+  });
+
+  it('links a free function the included header declares', () => {
+    expect(callsFrom('conformance/conformance_test_main.cc')).toContain('conformance::FailureCount (conformance/conformance_test.cc)');
+  });
+
+  it('links a test-named source file that a production file includes', () => {
+    expect(callsFrom('modules/expr.c')).toContain('run_fastjson_test (modules/fastjson_test.c)');
+  });
+
+  it('keeps it out of reach of a file that does not include it', () => {
+    expect(callsFrom('tools/report.cc').some((c) => c.includes('conformance_test.cc'))).toBe(false);
+  });
+});
+
+describe('a C/C++ test suite nothing in the program includes', () => {
+  it('is still out of reach of production code', () => {
+    expect(callsFrom('src/db.c').some((c) => c.includes('src/db_test.c'))).toBe(false);
+  });
+
+  it('does not implement the production header it includes to test', () => {
+    expect(callsFrom('src/hash.c').some((c) => c.includes('test/unit/hash.c'))).toBe(false);
+    // The test still reaches the code it tests.
+    expect(callsFrom('test/unit/hash.c')).toContain('hash (src/hash.c)');
+  });
+});

src/db/queries.ts

@@ -300,6 +300,7 @@ export class QueryBuilder {
     deleteRefsByRowIdsFull?: SqliteStatement;
     getAllFilePaths?: SqliteStatement;
     getAllNodeNames?: SqliteStatement;
+    getCppIncludeNodes?: SqliteStatement;
     getDominantFile?: SqliteStatement;
     getChangeStamp?: SqliteStatement;
     getTopRouteFile?: SqliteStatement;
@@ -3623,6 +3624,21 @@ export class QueryBuilder {
     return rows.map((r) => r.name);
   }
 
+  /**
+   * The import node of every C / C++ `#include`, narrowed to the path it
+   * spells and where it is written: the include graph reads nothing else, and
+   * decoding every import node of a large C++ project costs several times more.
+   */
+  getCppIncludeNodes(): Array<Pick<Node, 'id' | 'name' | 'filePath' | 'language' | 'startLine' | 'startColumn'>> {
+    if (!this.stmts.getCppIncludeNodes) {
+      this.stmts.getCppIncludeNodes = this.db.prepare(
+        `SELECT id, name, file_path AS filePath, language, start_line AS startLine, start_column AS startColumn
+           FROM nodes WHERE kind = 'import' AND language IN ('c', 'cpp')`
+      );
+    }
+    return this.stmts.getCppIncludeNodes.all() as Array<Pick<Node, 'id' | 'name' | 'filePath' | 'language' | 'startLine' | 'startColumn'>>;
+  }
+
   /**
    * Stream the distinct node names one row at a time — the incremental
    * counterpart to {@link getAllNodeNames} for callers that need to yield