Support BigQuery UNNEST WITH OFFSET, array OFFSET(), PARTITION BY expression, EXPORT/LOAD DATA, ASSERT and table snapshots - #2642
Open
hayssams wants to merge 8 commits into
Conversation
BigQuery addresses array elements positionally with OFFSET(n), ORDINAL(n), SAFE_OFFSET(n) and SAFE_ORDINAL(n). The latter three already parsed as functions; OFFSET is a reserved keyword and needed an explicit array-index rule. The accessor is kept in the AST as a Function so it deparses unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2
BigQuery writes UNNEST(array) [AS alias] WITH OFFSET [AS offsetAlias], where WITH OFFSET follows the table alias and carries an alias of its own. The pre-existing pre-alias form (WITH ORDINALITY|OFFSET AS t(a, b)) is untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2
BigQuery orders aggregate arguments as ARRAY_AGG(expr [IGNORE NULLS] [ORDER BY key] [LIMIT n]), so the ORDER BY may follow the null handling. Only the pre-null-handling position was accepted before. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2
BigQuery partitions by a bare expression rather than a partitioning method: PARTITION BY d, PARTITION BY DATE(ts), PARTITION BY RANGE_BUCKET(id, GENERATE_ARRAY(0, 100, 10)). Adds TablePartitioning.Type.EXPRESSION, which renders the expression without a method keyword or parentheses. The MySQL HASH/KEY/RANGE/LIST forms keep priority. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2
Adds ASSERT expression [AS description] as a statement, wired through the statement visitors, deparser and table-name finder. ASSERT stays a non-reserved keyword. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2
Adds EXPORT DATA [WITH CONNECTION connection] OPTIONS (option_list) AS query. Options are kept as an expression list, and the exported query is reachable from TablesNamesFinder. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2
Adds LOAD DATA {INTO | OVERWRITE} table [(column_definitions)]
[OPTIONS (option_list)] FROM FILES (file_options)
[WITH PARTITION COLUMNS] [WITH CONNECTION connection]. LOAD becomes a
non-reserved keyword so that a statement never starts with a bare
identifier, which would make statement starts ambiguous elsewhere.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2
BigQuery snapshots and clones a table with CREATE [SNAPSHOT] TABLE target CLONE source [FOR SYSTEM_TIME AS OF ...]. The clone source is kept on CreateTable and rendered by both toString() and CreateTableDeParser. SNAPSHOT itself already parsed as a create option. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2
Contributor
|
Run './gradlew spotlessApply' to fix all violations. |
hayssams
force-pushed
the
feature/bigquery-syntax
branch
from
September 15, 2026 14:13
6d597a2 to
eecd758
Compare
Contributor
Author
|
@manticore-projects |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds support for eight BigQuery (GoogleSQL) constructs that JSqlParser rejected. One commit per feature, each with tests that assert both the AST and a stable round-trip through
toString()and the deparser.Features, with the BigQuery documentation that specifies them
UNNEST(...) [AS alias] WITH OFFSET [AS offsetAlias]FROM UNNEST([1, 2, 3]) AS x WITH OFFSET AS posOFFSET(n)SELECT arr[OFFSET(0)]ORDER BYafterIGNORE NULLSin aggregatesARRAY_AGG(x IGNORE NULLS ORDER BY x LIMIT 5)PARTITION BY <expression>inCREATE TABLEPARTITION BY DATE(ts)ASSERT expression [AS description]ASSERT (SELECT COUNT(*) FROM t) > 0 AS 'empty'EXPORT DATA [WITH CONNECTION c] OPTIONS (...) AS queryEXPORT DATA OPTIONS (uri = 'gs://b/*.csv') AS SELECT * FROM tLOAD DATA {INTO | OVERWRITE} ... FROM FILES (...)LOAD DATA INTO ds.t FROM FILES (format = 'AVRO')CREATE [SNAPSHOT] TABLE target CLONE sourceCREATE SNAPSHOT TABLE ds.snap CLONE ds.t FOR SYSTEM_TIME AS OF ...Notes
ASSERT,EXPORT DATAandLOAD DATAare newStatementtypes, wired throughStatementVisitor,StatementVisitorAdapter,StatementDeParser,TablesNamesFinder,StatementFeatureVisitorandStatementValidator.TablesNamesFinderreaches the exported query and the load target.CLONEis kept onCreateTableand rendered by bothtoString()andCreateTableDeParser.PARTITION BY <expression>is a newTablePartitioning.Type.EXPRESSION; the MySQLHASH/KEY/RANGE/LISTforms keep priority.ASSERT,CLONE,LOAD) are non-reserved and stay usable as identifiers; regression tests cover that.Test plan
BigQueryTest, including negative/regression cases (LIMIT ... OFFSET,WITH ORDINALITY AS t(a, b),PARTITION BY HASH (id),SELECT assert FROM t).SpecialOracleTest.testAllSqlsParseDeparse, which fails on an unmodifiedmasterin this environment too.0 errors and 1 warnings, the same warning as onmaster.🤖 Generated with Claude Code
https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2