Skip to content

SQL relation nodes: vector-search coverage, write-lineage, and data-aware tool descriptions #1755

Description

@DeusData

#1749 landed first-class Table / View nodes with FROM/JOIN lineage edges (distilled from #582, closing #574). Three integration pieces were deliberately deferred out of that change to keep it atomic. Recording them here so they are not lost.

1. Vector / semantic search does not reach relation nodes

store.c's vector query filters on CBM_SQL_CALLABLE_OR_TYPE_LABELS, and relations were deliberately not added there: the embedding pass does not currently produce vectors for Table/View nodes, so widening the filter would only add rows that can never match. The work is the embedding side, not the query side.

Result today: a semantic question like "where is customer data stored" will not surface a Table node, although BM25 search and get_architecture both do.

2. Write-lineage is missing

Only read lineage exists (view -USAGE-> table, from FROM/JOIN). INSERT INTO, UPDATE, MERGE, and CREATE TABLE AS SELECT targets produce no edge, so "what writes this table" is unanswerable. The natural shape is a WRITES edge from the enclosing statement/procedure to the relation, reusing the same collect_sql_relation_usages walk with a direction flag.

Note the constraint: the relation veto in cbm_registry_resolve means any new consumer must opt in explicitly via cbm_registry_resolve_lineage, which is intentional.

3. MCP tool descriptions are code-oriented

Raised by @krahnikblis on #582: the tool descriptions speak entirely in terms of functions, classes, and calls, so an agent holding a graph that now contains tables and views has no cue that data questions are answerable. Wording only — no behaviour change.

Out of scope here

dbt manifest ingestion and dbt Jinja lineage (#583 / #584) are a separate direction question and are tracked on those PRs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestparsing/qualityGraph extraction bugs, false positives, missing edges

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions