Expose structured ORDER BY fragment parsing

[minleejae]

Consumers that assemble queries from independent sort specifications currently need to wrap fragments in a SELECT or retain them as opaque expressions. For example, DBeaver's ORDER BY handling uses CustomExpression for a sort fragment (pinned consumer code). A fragment API exposes the expression and sort flags directly, so callers can inspect and rewrite them through the ordinary AST.

Add CCJSqlParserUtil.parseOrderByElements(String) and its parser-configuration overload. Input such as COALESCE(score, 0) DESC NULLS LAST, id ASC produces a List<OrderByElement>; the input excludes the ORDER BY keywords. Null or empty input returns an empty list, and trailing tokens are rejected.

The existing ORDER BY clause now delegates to a shared element-list production. Column-type parsing and the new API also share fragment parser setup, fast/complex retry, full-input checking, and exception handling. Existing column-type behavior is preserved, and each invocation starts with fresh parser state.

Validation: full Java 17 Gradle check passes, including the zero-conflict JavaCC grammar gate, tests, formatting, Checkstyle, PMD, and SpotBugs. Tests compare fragments with complete ORDER BY clauses, exercise configured quoting and parameter indices, verify custom expression deparsing and round trips, and reject incomplete or trailing SQL.

[manticore-projects]

Mhhh, I don't like this. I believe I do understand the requirement: simpler access to certain fragments, the walking the AST is cumbersome. But instead of specific solutions, I wonder if a kind of "querying" might be the better and more comprehensive solution here. Like XSL for an XML, something like */WHERE[1]/EXPRESSION[*]. In fact, I drafted something crude like this in JSQLFormatter years ago.

Suggestion: lets put this on small flame for a while and focus on getting a 5.4 release out. And then craft a proper AST querying mechanism for this together.
(Unless it was really urgent for you, then we would just take that in and deprecate it later. Just let me know.)

[minleejae]

@manticore-projects
This isn't urgent for me. I agree that a general-purpose AST querying mechanism would be more useful in the long term, and I'd be happy to work on that together. Let's leave this PR on hold and focus on the 5.4 release first.

[minleejae]

@manticore-projects Following up on the AST querying idea, I compared 59 DDL/expression cases on 5.4 and current master (6312f9e5). Some consumer-side scanning comes from information that is still unstructured:

• ColumnOption already structures defaults, references and identity declarations, but column COLLATE, COMMENT, ON UPDATE, generated expressions and several flags remain raw tokens.
• CURRENT_TIMESTAMP, CURRENT_TIMESTAMP(6) and bare LOCALTIME produce TimeKeyExpression, Function and Column respectively. A common temporal representation needs to preserve dialect rules and quoted/qualified identifier behavior.
• DROP INDEX x ON app.t stores the owner in Drop.getParameters() as [ON, app.t]. There is no structured owner table.
• ColDataType exposes precision/scale, but getDataType() can still contain the parameters, e.g. decimal (10, 2), so extracting the base type still requires string handling.

These seem complementary to AST querying: enriching the model would give the querying layer structured fields to traverse. I suggest handling them in focused PRs that preserve existing accessors and rendering, while keeping general selection/traversal design in this discussion. The quoted-name and negative-scale parser fixes are already in #2638.

Does that separation fit your plan, or are there node/property conventions you would like those AST additions to follow for the future querying API?

[manticore-projects]

I spent some tokens quickly to mock something:

get me the where expression of the second union query of the 3rd with clause becomes a simple path /withItemsList[2]/select/selects[1]/where:

Expression w = AstPath.get(root, "/withItemsList[2]/select/selects[1]/where", Expression.class);

Three variants, all verified against the mock:

/withItemsList[2]/select/selects[1]/where — positional, your literal phrasing
/withItemsList[name='w2']/select/selects[1]/where — by CTE name, survives reordering
//SetOperationList/selects[1]/where — shape-agnostic, doesn't care how deep the union sits

Main problem is: our path expressions would be unstable. Any API change and a formerly working path possible won't work. Also there is no schema to verify a path against. Maybe we would need to create such schemas using JAXB. Also, there's no sibling axis, so "the union branch after the one matching X" isn't expressible. We would need to add following-sibling/preceding-sibling yet.

AstPath.zip

[manticore-projects]

Does that separation fit your plan, or are there node/property conventions you would like those AST additions to follow for the future querying API?

In my opinion, developing a generic AST Path Query mechanism and enriching the API where needed are two different things. DDL won't have much benefit from our AST Query because they are not nested (unless you really want to stress it). But Queries can be deeply nested structures and here AST Path can make a lot of sense.