Repository navigation
Conversation
sqlglot 30.22.0 removed `infer_schema` from the optimizer (tobymao/sqlglot#8552). With an empty schema, `qualify` no longer attributes an unqualified column to the scope's only source without known columns, so `cll()` raised "Column 'x' could not be resolved". In recce that hit models that are in the catalog but have no resolvable dbt parents (hard-coded relations), whose columns fell back to `unknown`. When the schema is empty, `cll()` now qualifies with validation off, re-applies the removed attribution, then validates. A non-empty schema goes through `qualify` unchanged, matching sqlglot <30.22 exactly. Also lift the `sqlglot<30.22` cap to `<31.0` (lock: 30.24.0) and bump the CLL cache version so results cached under 30.22-30.23 are dropped. Closes DRC-4901 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jared Scott <jared.scott@datarecce.io>
Codecov Report❌ Patch coverage is
... and 4 files with indirect coverage changes 🚀 New features to boost your workflow:
|
gcko
deleted the
feature/drc-4901-adapt-column-level-lineage-to-sqlglot-3022-and-lift-the
branch
October 21, 2026 02:35
2 of 4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR checklist
What type of PR is this?
fix
What this PR does / why we need it:
sqlglot 30.22.0 removed
infer_schemafrom the optimizer (tobymao/sqlglot#8552). Before that, when no schema was given,qualifyattributed an unqualified column to the one source in the scope with no known columns. Now it raisesOptimizeError: Column 'x' could not be resolved. PR #1573 cappedsqlglot<30.22as a stopgap.This PR adapts
cll()and lifts the cap:recce/util/cll.py: new_qualify()helper. A non-empty schema goes throughqualifyunchanged. With an empty schema it qualifies withvalidate_qualify_columns=False, re-applies the removed attribution (_attribute_to_schemaless_source), then callsvalidate_qualify_columns. This is exact parity with sqlglot <30.22, which also inferred only when the schema was empty.Resolvermoved fromqualify_columnsto its own module in late 2025, so the import falls back for the>=27.29.0floor._CACHE_SCHEMA_VERSION3 → 4, sounknownresults cached by users on 30.22–30.23 are dropped. Cache eviction is access-based, so hot entries would otherwise stay stale.pyproject.toml:sqlglot>=27.29.0,<31.0;uv.lockresolves 30.24.0.Which issue(s) this PR fixes:
Closes DRC-4901
Special notes for your reviewer:
DbtAdapter,cll()only gets an empty schema for a model that is incatalog.jsonbut has no resolvable dbt parents, for example one that hard-codesfrom raw_events. The new adapter testtest_model_reading_hardcoded_relationcovers this; it reportsunknownwithout the fix andrenamedwith it. Without a catalog,get_cllreturns before callingcll(), so that path is unaffected.schema[parent] = {}, which raisesSchemaErroron every sqlglot version. That's a separate, older gap, tracked in DRC-4918.{}(SchemaError), and{"*": ...}still fails to resolve.test_cll.py,test_dbt_cll.pyandtest_breaking.pypass on sqlglot 27.29.0, 30.21.0 and 30.24.0. The full suite on 30.24.0 gave 2040 passed, 5 skipped.recce/util/change_classifier.pyalso callsqualify, but only with a non-empty schema, so it never relied on the removed inference. Its tests pass unchanged on 30.24...._ambiguous_without_schema,..._with_partial_schema) pass on main too. They pin parity so the attribution can't spread to ambiguous or partial-schema cases.Does this PR introduce a user-facing change?:
🤖 Generated with Claude Code