Repository navigation
Conversation
…mn and function call Remove errorCapturingIdentifierExtra (MINUS identifier)+ rule from SparkSqlParser.g4 so SLL prediction does not misparse column-minus-function expressions as hyphenated identifiers. Co-authored-by: Cursor <cursoragent@cursor.com>
Collaborator
|
好的,PR已收到,review没问题后会随着下个版本一起发布,感谢你的关注 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #501
根因分析
SparkSqlParser.g4沿用了 Apache Spark 官方语法的errorCapturingIdentifier(含errorCapturingIdentifierExtra : (MINUS identifier)+)。在 Apache Spark 中,该规则配合 ScalaPostProcessor用于捕获未加反引号的连字符标识符(如test-table)并抛出专用异常;但本仓库在 #163 导入并精简语法标签时移除了#errorIdent,且从未实现对应的后置报错逻辑。后续在 #179、#218、#358 中,
lateralView(viewName)、namedExpression(columnName)和primaryExpression(columnNamePath)先后改为引用multipartIdentifier -> errorCapturingIdentifier,打破了原语法“不要在紧跟表达式的位置使用errorCapturingIdentifier”的约束。在
PredictionMode.SLL(#269)下解析SELECT (price - if(...)) * quantity FROM t;时,由于 SLL 不保留外层调用栈,errorCapturingIdentifierExtra的Alt 1(将price - if吞作单个标识符)借由lateralView规则消化了后续的(...),并在外层右括号)处与Alt 2(正常算术减法)产生 SLL 完备冲突。ANTLR 默认选择编号更小的Alt 1,导致price - if被误判为列名,进而对随后的(报错missing ')' at '('。改动内容
src/grammar/spark/SparkSqlParser.g4中errorCapturingIdentifierExtra的(MINUS identifier)+错误捕获分支,将errorCapturingIdentifier简化为直接匹配identifier(与 #432 移除 PostgreSQL 错误捕获规则一致),并重新生成src/lib/spark下的解析器代码。test/parser/spark下关于列与函数相减表达式语法校验、减法表达式实体收集(getAllEntities)、SET/RESET带中划线配置项以及非法连字符标识符校验的测试用例。影响范围与风险评估
SparkSqlParser.g4的errorCapturingIdentifier规则,可随时通过单次提交回滚。validate):修复括号内列与函数调用相减(如SELECT (price - if(...)) * quantity FROM t;)、无FROM子句的列与函数调用相减(如SELECT price - if(...);)以及无参函数减法(如SELECT a - f();)的语法校验误报。getAllEntities):修复SELECT a - b FROM t;、SELECT a - f(x) FROM t;等减法表达式被静默归类为字面量列(ColumnDeclareType.LITERAL)而非表达式列(ColumnDeclareType.EXPRESSION)的问题。-语法的兼容性:SET/RESET配置语句(如SET tez.grouping.max-size = 1;、SET -v;、RESET tez.grouping.max-size;):走#setAny(KW_SET .*?)和#resetAny(KW_RESET .*?),不经过errorCapturingIdentifier,不受影响。TBLPROPERTIES ('max-age' = '1'))、负数常量、时间间隔INTERVAL -1 DAY、反引号包裹的带中划线标识符(如`test-table`、`col-a`):不受影响。CREATE DATABASE test-db;、CREATE TABLE test-tbl (id-col INT);):将按 Spark SQL 规范正常拦截并报语法错误。