fix(go): a type declared on its own keeps its doc comment

CHANGELOG.md

@@ -39,6 +39,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 - In Go, an exported type written without a package, like `Node` in `func Walk(v Visitor, node Node)`, now links to the type of that name in its own package, whether it is a struct, an interface or another kind of type. Before, it could link to a struct of the same name in an unrelated package, so prometheus's PromQL parser functions pointed at the Kubernetes discovery `Node` struct instead of the parser's own `Node` interface, or, for a result type like `Appender` in `func (f *fanout) Appender(…) Appender`, to the method itself. A type from a package outside your project, like `apiv1.Node` or `http.Handler`, no longer links to a struct of the same name in the file that uses it; a variadic parameter like `...storage.Filter` now links to that package's type; a route handler written as a method value, like `h.Follow` in an Echo or Gin app, links to that method rather than a model struct named `Follow`; and a method called on another call's result or on an indexed value, like `err[i].Error()` or a chained `.String()`, is no longer recorded as creating a struct of the same name. Re-index Go projects after upgrading.
 - In Go, a function or method passed as a value, like `sync.Pool{New: wm.new}`, now shows the function that passes it among its callers and impact even when it is named `new`, `nil`, `None`, `self` or a handful of similar words, or when a line break or comment sits beside the dot. Before, such a value was skipped, so a method used only that way looked unused. Re-index Go projects after upgrading.
 - In Go, a type in a parameter, a result or a composite literal now always links to a type, never to a method or function that shares its name. Before, an unexported type like `keyIndex` in etcd's `func (ti *treeIndex) KeyIndex(keyi *keyIndex) *keyIndex` linked to the `keyIndex` method declared beside it, prometheus's `samples{…}` literals to a `samples` method, and a type from a package outside your project, like `config.URL` in `&config.URL{…}`, to whatever project method had that name. A type written without a package, including an unexported one or one named like `AppenderV2` or `Histogram_CountInt`, now links to its own package's type when there is one, and a generic method's type parameter, like `T` in `func (p *Pool[T]) Get() T`, no longer links to an unrelated `T`. Re-index Go projects after upgrading.
+- In Go, a type declared on its own, like `type Point struct{…}` under a `// Point is …` comment, now keeps that comment as its documentation, as a type inside a `type ( … )` group already did, so searching for what a type does can find it. A comment at the end of a line of code, like `const sides = 4 // sides of a square.`, is no longer taken as the documentation of the declaration under it. Re-index Go projects after upgrading.
 - Indexing a project that includes large bundled JavaScript files, such as a copy of pdf.js or d3, is fast again: since 1.6.2, resolving the calls in a JavaScript or TypeScript file re-read the file's text above each call, so a single bundled library could add many seconds to an index. The graph it builds is unchanged. Thanks @bompus for the report. (#2334)
 - Indexing a project that ships a large minified JavaScript file, such as the GraphiQL bundle inside go-ethereum, no longer stalls at "Resolving refs" when CodeGraph runs on Node 22, and linking such a file is faster on any Node version. The graph it builds is unchanged.
 - The time `codegraph init` and `codegraph index` print beside the node and edge counts now covers the whole run, resolving references and linking included, and `codegraph sync` reports its whole run the same way. Before, it counted only reading and parsing the files, which can be a small part of an index, so a slow index looked fast. Thanks @bompus for the report. (#2334)

__tests__/fixtures/kernel-parity/torture.go

@@ -6,14 +6,15 @@ import (
 	pkga "example.com/other/pkga"
 )
 
-const MAX_ITEMS = 128
+const MAX_ITEMS = 128 // the line's own comment, not DefaultRegistry's doc
 
 var DefaultRegistry = NewRegistry()
 
 var handlerTable = map[string]func(int){
 	"recv": TargetCb,
 }
 
+// Widget is documented above its own type declaration.
 type Widget struct {
 	*Base
 	Queryable
@@ -28,6 +29,13 @@ type Stack[T any] struct {
 	items []T
 }
 
+// Units of time: the group's comment, no member's doc.
+type (
+	Seconds int
+	// Minutes is documented inside its group.
+	Minutes int
+)
+
 type Core interface {
 	Reader
 	pkga.Closer // qualified

__tests__/go-type-doc-comments.test.ts

@@ -0,0 +1,220 @@
+/**
+ * A Go type declared on its own lost its doc comment in both extractors:
+ *
+ *   // Foo is documented.
+ *   type Foo struct{}
+ *
+ * tree-sitter-go wraps every type in a `type_declaration`, and the comment is
+ * that declaration's previous sibling. The node is made from the spec inside
+ * it (a `type_spec`, or a `type_alias` for `type A = B`), whose only
+ * predecessor is the `type` keyword, so the docstring walk found nothing. A
+ * type inside a `type ( … )` group kept its comment, which sits beside it in
+ * the parentheses.
+ *
+ * A declaration holding one spec is now read from outside, as go doc reads
+ * it. A group's leading comment stays the group's: it documents the group,
+ * not its first member.
+ *
+ * Reading from outside the declaration exposed a gap Go functions already
+ * had: a comment written after code on the line above (`const sides = 4 //
+ * sides of a square.`) belongs to that line, not to the declaration below
+ * it, and the docstring now starts after it.
+ *
+ * Runs against the native kernel (when built) and the wasm extractor, which
+ * must agree.
+ */
+import { describe, it, expect, beforeAll, beforeEach, afterEach } from 'vitest';
+import * as fs from 'fs';
+import * as path from 'path';
+import { extractFromSource } from '../src/extraction';
+import { initGrammars, loadGrammarsForLanguages } from '../src/extraction/grammars';
+import { tryKernelExtract, resetKernelForTests } from '../src/extraction/kernel';
+import type { ExtractionResult } from '../src/types';
+
+const KERNEL_PATH = path.join(
+  __dirname,
+  '..',
+  'codegraph-kernel',
+  'prebuilds',
+  `${process.platform}-${process.arch}`,
+  'codegraph-kernel.node'
+);
+const kernelAvailable = fs.existsSync(KERNEL_PATH) || process.env.CODEGRAPH_KERNEL_EXPECT === '1';
+
+const SOURCE = `// Copyright 2026 The Shapes Authors.
+
+// Package shapes is documented.
+package shapes
+
+type Bare struct{}
+
+// Point is documented on its own.
+type Point struct{ X, Y int }
+
+// Shape is an interface documented on its own.
+type Shape interface {
+	Area() float64
+}
+
+// ID is a defined type documented on its own.
+type ID int
+
+// Set is a generic type documented on its own.
+type Set[T comparable] map[T]struct{}
+
+// Alias is an alias documented on its own.
+type Alias = Point
+
+/*
+Box is documented in a block comment.
+*/
+type Box struct{}
+
+// Kinds of shapes. The comment documents the group.
+type (
+	Circle struct{}
+
+	// Square is documented inside the group.
+	Square struct{}
+)
+
+// Lone is the only member of its group.
+type (
+	Lone int
+)
+
+// A group of one whose member has its own comment.
+type (
+	// Own is documented inside its group.
+	Own int
+)
+
+// Pair groups a type with its alias.
+type (
+	First  int
+	Second = First
+)
+
+type (
+	Celsius float64 // degrees C.
+	Kelvin  float64
+)
+
+// sides is documented above its line.
+const sides = 4 /* four */ // sides of a square.
+
+// Rect is documented below a constant's line.
+type Rect struct{}
+
+var origin = Point{} // the origin.
+type Polygon struct{}
+
+// Area is documented as before.
+func Area(s Shape) float64 { return s.Area() }
+
+// Norm is documented as before.
+func (p Point) Norm() int { return p.X } // Norm's line comment.
+func Next() int { return 0 }
+`;
+
+const ENV_KEYS = ['CODEGRAPH_KERNEL', 'CODEGRAPH_KERNEL_LANGS'] as const;
+
+describe('Go doc comments on types declared on their own', () => {
+  let savedEnv: Record<string, string | undefined> = {};
+
+  beforeAll(async () => {
+    await initGrammars();
+    await loadGrammarsForLanguages(['go']);
+  });
+
+  beforeEach(() => {
+    savedEnv = Object.fromEntries(ENV_KEYS.map((k) => [k, process.env[k]]));
+    resetKernelForTests();
+  });
+
+  afterEach(() => {
+    for (const k of ENV_KEYS) {
+      if (savedEnv[k] === undefined) delete process.env[k];
+      else process.env[k] = savedEnv[k];
+    }
+    resetKernelForTests();
+  });
+
+  function extract(backend: 'kernel' | 'wasm', source: string): ExtractionResult {
+    if (backend === 'wasm') {
+      process.env.CODEGRAPH_KERNEL = '0';
+      return extractFromSource('shapes/shapes.go', source, 'go');
+    }
+    delete process.env.CODEGRAPH_KERNEL;
+    process.env.CODEGRAPH_KERNEL_LANGS = 'all';
+    const result = tryKernelExtract('shapes/shapes.go', source, 'go');
+    expect(result, 'kernel extraction').not.toBeNull();
+    return result!;
+  }
+
+  const backends = kernelAvailable ? (['kernel', 'wasm'] as const) : (['wasm'] as const);
+
+  for (const crlf of [false, true]) {
+    it.each(backends)(`%s${crlf ? ' (CRLF)' : ''}`, (backend) => {
+      const result = extract(backend, crlf ? SOURCE.replace(/\n/g, '\r\n') : SOURCE);
+      const doc = (kind: string, name: string): string | null => {
+        const found = result.nodes.filter((n) => n.kind === kind && n.name === name);
+        expect(found, `${kind} ${name}`).toHaveLength(1);
+        return found[0].docstring ?? null;
+      };
+
+      expect({
+        // A type declared on its own takes the comment above `type`, and the
+        // file's header and package comment stay before the package clause.
+        Bare: doc('struct', 'Bare'),
+        Point: doc('struct', 'Point'),
+        Shape: doc('interface', 'Shape'),
+        ID: doc('type_alias', 'ID'),
+        Set: doc('type_alias', 'Set'),
+        Alias: doc('type_alias', 'Alias'),
+        Box: doc('struct', 'Box'),
+        // A group's comment is the group's; a member's own comment is found
+        // beside it. A group of one is that type's declaration, as in go doc,
+        // unless its member has a comment of its own.
+        Circle: doc('struct', 'Circle'),
+        Square: doc('struct', 'Square'),
+        Lone: doc('type_alias', 'Lone'),
+        Own: doc('type_alias', 'Own'),
+        // An alias is one of a group's members too.
+        First: doc('type_alias', 'First'),
+        Second: doc('type_alias', 'Second'),
+        // A comment after code is that line's, whatever comes below it.
+        Celsius: doc('type_alias', 'Celsius'),
+        Kelvin: doc('type_alias', 'Kelvin'),
+        sides: doc('constant', 'sides'),
+        Rect: doc('struct', 'Rect'),
+        Polygon: doc('struct', 'Polygon'),
+        Area: doc('function', 'Area'),
+        Norm: doc('method', 'Norm'),
+        Next: doc('function', 'Next'),
+      }).toEqual({
+        Bare: null,
+        Point: 'Point is documented on its own.',
+        Shape: 'Shape is an interface documented on its own.',
+        ID: 'ID is a defined type documented on its own.',
+        Set: 'Set is a generic type documented on its own.',
+        Alias: 'Alias is an alias documented on its own.',
+        Box: 'Box is documented in a block comment.',
+        Circle: null,
+        Square: 'Square is documented inside the group.',
+        Lone: 'Lone is the only member of its group.',
+        Own: 'Own is documented inside its group.',
+        First: null,
+        Second: null,
+        Celsius: null,
+        Kelvin: null,
+        sides: 'sides is documented above its line.',
+        Rect: "Rect is documented below a constant's line.",
+        Polygon: null,
+        Area: 'Area is documented as before.',
+        Norm: 'Norm is documented as before.',
+        Next: null,
+      });
+    });
+  }
+});

codegraph-kernel/src/docstring.rs

@@ -142,6 +142,24 @@ pub fn preceding_docstring_stepping_over(
     node: Node,
     src: &str,
     step_over: &[&str],
+) -> Option<String> {
+    preceding_docstring_with(node, src, step_over, false)
+}
+
+/// getPrecedingDocstring's `skipTrailing` (Go) — the comments the run opens
+/// with that belong to the line above are left out: one written after code on
+/// its line (`const n = 4 // four.`), and any that begins on the line such a
+/// comment ends. Go reads them as that line's comment, never the next
+/// declaration's doc.
+pub fn preceding_docstring_skipping_trailing(node: Node, src: &str) -> Option<String> {
+    preceding_docstring_with(node, src, &[], true)
+}
+
+fn preceding_docstring_with(
+    node: Node,
+    src: &str,
+    step_over: &[&str],
+    skip_trailing: bool,
 ) -> Option<String> {
     let mut anchor = node;
     while let Some(parent) = anchor.parent() {
@@ -152,33 +170,57 @@ pub fn preceding_docstring_stepping_over(
         }
     }
 
-    let mut comments: Vec<&str> = Vec::new();
+    let mut comments: Vec<Node> = Vec::new();
     let mut sibling = anchor.prev_named_sibling();
     while let Some(s) = sibling {
         if is_comment(s.kind()) {
-            comments.push(&src[s.byte_range()]);
+            comments.push(s);
             sibling = s.prev_named_sibling();
         } else if step_over.contains(&s.kind()) {
             sibling = s.prev_named_sibling();
         } else {
             break;
         }
     }
-    if comments.is_empty() {
+    comments.reverse(); // collected nearest-first; TS unshifts to keep source order
+
+    let mut first = 0;
+    if skip_trailing {
+        let mut end_row = 0;
+        for c in &comments {
+            let trails = if first == 0 {
+                follows_code_on_its_line(*c, src)
+            } else {
+                c.start_position().row == end_row
+            };
+            if !trails {
+                break;
+            }
+            end_row = c.end_position().row;
+            first += 1;
+        }
+    }
+    if first == comments.len() {
         return None;
     }
-    comments.reverse(); // collected nearest-first; TS unshifts to keep source order
     Some(
-        comments
+        comments[first..]
             .iter()
-            .map(|c| clean_comment_markers(c))
+            .map(|c| clean_comment_markers(&src[c.byte_range()]))
             .collect::<Vec<_>>()
             .join("\n")
             .trim()
             .to_string(),
     )
 }
 
+/// followsCodeOnItsLine (tree-sitter-helpers.ts) — whether code comes before
+/// `node` on the line it starts on.
+fn follows_code_on_its_line(node: Node, src: &str) -> bool {
+    let before = src[..node.start_byte()].trim_end_matches([' ', '\t']);
+    !before.is_empty() && !before.ends_with(['\n', '\r'])
+}
+
 #[cfg(test)]
 mod tests {
     use super::*;
@@ -231,4 +273,30 @@ mod tests {
             Some("a\nb")
         );
     }
+
+    /// Skipping trailing comments leaves out one written after code on its
+    /// line and one beginning on the line it ends; a comment on a line of its
+    /// own still opens the run. Without it, nothing is left out.
+    #[test]
+    fn skips_the_comments_that_trail_code() {
+        let grammar = crate::langs::grammar_for("go").expect("go grammar");
+        let mut parser = tree_sitter::Parser::new();
+        parser.set_language(&grammar).unwrap();
+        let src = "package p\n\nconst n = 4 /* a */ // b\n// c\nfunc f() {}\n\nvar m = 1 // d\nfunc g() {}\n";
+        let tree = parser.parse(src, None).unwrap();
+        let root = tree.root_node();
+        let func = |name: &str| {
+            (0..root.named_child_count())
+                .filter_map(|i| root.named_child(i))
+                .find(|n| {
+                    n.kind() == "function_declaration"
+                        && n.child_by_field_name("name").map(|id| &src[id.byte_range()]) == Some(name)
+                })
+                .expect(name)
+        };
+        assert_eq!(preceding_docstring(func("f"), src).as_deref(), Some("a\nb\nc"));
+        assert_eq!(preceding_docstring_skipping_trailing(func("f"), src).as_deref(), Some("c"));
+        assert_eq!(preceding_docstring(func("g"), src).as_deref(), Some("d"));
+        assert_eq!(preceding_docstring_skipping_trailing(func("g"), src), None);
+    }
 }