[FLINK-39780] Make TABLE keyword optional in LATERAL context by fhueske · Pull Request #28276 · apache/flink

fhueske · 2026-05-29T08:34:42Z

What is the purpose of the change

Extend the SQL parser accept the implicit table-function-call form (e.g. "FROM t, LATERAL fn(args)") in addition to the explicit "LATERAL TABLE(fn(args))" form.
Mirrors FLINK-36824, which made the TABLE wrapper optional outside LATERAL.

Brief change log

adjust Parser.jj to accept table function calls in LATERAL without TABLE keyword.
- This is another optional parsing path and does not affect existing paths

Verifying this change

adjusted Calcite's unit tests in FlinkSqlParserImplTest (by overwriting them) to fix the error positions
- the new parsing rule cause the error positions to slightly shift
added unit tests in FlinkSqlParserImplTest

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): yes
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? yes
If yes, how is the feature documented? yes

Documentation is adjusted. We now use the shorter, implicit syntax for table function calls in LATERAL.
The SQL syntax overview is adjusted to cover both cases (LATERAL with and without TABLE keyword).
Also extended the overview with syntax for non-LATERAL table function calls with and without TABLE keyword which was missing so far.
I double-checked that the AI-generated Chinese text corresponds to the English version using Google Translate 🤖

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Generated-by: Claude Code 2.1.148 (Opus 4.7)

Extend the SQL parser accept the implicit table-function-call form (e.g. "FROM t, LATERAL fn(args)") in addition to the explicit "LATERAL TABLE(fn(args))" form. Mirrors FLINK-36824, which made the TABLE wrapper optional outside LATERAL. Add tests in FlinkSqlParserImplTest and overwrites methods in Calcite's SqlParserTest that need adjustments.

…nction calls

flinkbot · 2026-05-29T08:43:08Z

CI report:

f71f7a8 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

@flinkbot run azure re-run the last Azure build

snuyanzin · 2026-05-29T09:53:42Z

+    |
+        // LATERAL with implicit table function call syntax,
+        // e.g. "FROM t, LATERAL fn(...)" instead of "FROM t, LATERAL TABLE(fn(...))".
+        // The non-LATERAL implicit form is handled by the CompoundTableIdentifier
+        // branch above.
+        LOOKAHEAD(2)
+        <LATERAL> { lateral = true; }
+        tableName = CompoundTableIdentifier()
+        tableRef = ImplicitTableFunctionCallArgs(tableName)
+        tableRef = addLateral(tableRef, lateral)


this is the Calcite file from https://github.com/apache/calcite/blob/main/core/src/main/codegen/templates/Parser.jj

The only reason we have it here is a number of backports from Calcite.

In case we need to make changes here: ideally first need to make them in Calcite and then backport here

Ideally this file should be removed once we update Calcite to versions containing all the backports

Agree this is danger zone ⚠️

raminqaf

Thanks for the contribution @fhueske left some comments.

raminqaf · 2026-05-29T12:37:11Z

    [ TABLE ] tablePath [ dynamicTableOptions ] [systemTimePeriod] [[AS] correlationName]
-  | LATERAL TABLE '(' functionName '(' expression [, expression ]* ')' ')'
+  | [ LATERAL ] functionName '(' expression [, expression ]* ')'
+  | [ LATERAL ] TABLE '(' functionName '(' expression [, expression ]* ')' ')'


If table is optional

Suggested change

| [ LATERAL ] TABLE '(' functionName '(' expression [, expression ]* ')' ')'

| [ LATERAL ] [TABLE] '(' functionName '(' expression [, expression ]* ')' ')'

raminqaf · 2026-05-29T12:38:07Z

 在 Table API 中，表值函数是通过 `.joinLateral(...)` 或者 `.leftOuterJoinLateral(...)` 来使用的。`joinLateral` 算子会把外表（算子左侧的表）的每一行跟跟表值函数返回的所有行（位于算子右侧）进行 （cross）join。`leftOuterJoinLateral` 算子也是把外表（算子左侧的表）的每一行跟表值函数返回的所有行（位于算子右侧）进行（cross）join，并且如果表值函数返回 0 行也会保留外表的这一行。

-在 SQL 里面用 `JOIN` 或者 以 `ON TRUE` 为条件的 `LEFT JOIN` 来配合 `LATERAL TABLE(<TableFunction>)` 的使用。
+在 SQL 里面用 `JOIN` 或者 以 `ON TRUE` 为条件的 `LEFT JOIN` 来配合 `LATERAL <TableFunction>(...)` 的使用（等价的写法为 `LATERAL TABLE(<TableFunction>(...))`）。


AFAK we only add the english text as place holder and let folks translate it to Chinese

raminqaf · 2026-05-29T12:38:23Z

    [ TABLE ] tablePath [ dynamicTableOptions ] [systemTimePeriod] [[AS] correlationName]
-  | LATERAL TABLE '(' functionName '(' expression [, expression ]* ')' ')'
+  | [ LATERAL ] functionName '(' expression [, expression ]* ')'
+  | [ LATERAL ] TABLE '(' functionName '(' expression [, expression ]* ')' ')'


Suggested change

| [ LATERAL ] TABLE '(' functionName '(' expression [, expression ]* ')' ')'

| [ LATERAL ] [TABLE] '(' functionName '(' expression [, expression ]* ')' ')'

raminqaf · 2026-05-29T12:39:11Z

+    |
+        // LATERAL with implicit table function call syntax,
+        // e.g. "FROM t, LATERAL fn(...)" instead of "FROM t, LATERAL TABLE(fn(...))".
+        // The non-LATERAL implicit form is handled by the CompoundTableIdentifier
+        // branch above.
+        LOOKAHEAD(2)
+        <LATERAL> { lateral = true; }
+        tableName = CompoundTableIdentifier()
+        tableRef = ImplicitTableFunctionCallArgs(tableName)
+        tableRef = addLateral(tableRef, lateral)


Agree this is danger zone ⚠️

raminqaf · 2026-05-29T12:40:03Z

+    /**
+     * Overrides the parent {@link org.apache.calcite.sql.parser.SqlParserTest#testLateral()}
+     * because Flink makes the {@code TABLE} keyword optional inside {@code LATERAL}. With this
+     * change, {@code LATERAL <identifier>} is the start of an implicit table-function call; the
+     * parser expects an argument list next, so the error position shifts.
+     */


Do we really need so many javadocs and comments on tests? Claude loves to do it but IMO test should be clean enough to read with a good function name

raminqaf · 2026-05-29T12:40:41Z

+     * unexpected token ({@code for}).
+     */
+    @Test
+    void testTemporalTable() {


Seems like a good candidate for parasitized tests. WDYT?

fhueske added 2 commits May 29, 2026 10:21

[FLINK-39780][docs] Adjust documentation to implicit LATERAL table fu…

f71f7a8

…nction calls

snuyanzin reviewed May 29, 2026

View reviewed changes

raminqaf reviewed May 29, 2026

View reviewed changes

github-actions Bot added the community-reviewed PR has been reviewed by the community. label May 29, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FLINK-39780] Make TABLE keyword optional in LATERAL context#28276

[FLINK-39780] Make TABLE keyword optional in LATERAL context#28276
fhueske wants to merge 2 commits into
apache:masterfrom
confluentinc:fhueske-FLINK-39780-Make-TABLE-keyword-optional-in-LATERAL-context

fhueske commented May 29, 2026

Uh oh!

flinkbot commented May 29, 2026 •

edited

Loading

Uh oh!

snuyanzin May 29, 2026

Uh oh!

snuyanzin May 29, 2026

Uh oh!

raminqaf May 29, 2026

Uh oh!

raminqaf left a comment

Uh oh!

raminqaf May 29, 2026

Uh oh!

raminqaf May 29, 2026

Uh oh!

raminqaf May 29, 2026

Uh oh!

raminqaf May 29, 2026

Uh oh!

raminqaf May 29, 2026

Uh oh!

raminqaf May 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	\| [ LATERAL ] TABLE '(' functionName '(' expression [, expression ]* ')' ')'
	\| [ LATERAL ] [TABLE] '(' functionName '(' expression [, expression ]* ')' ')'

Conversation

fhueske commented May 29, 2026

What is the purpose of the change

Brief change log

Verifying this change

Does this pull request potentially affect one of the following parts:

Documentation

Was generative AI tooling used to co-author this PR?

Uh oh!

flinkbot commented May 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI report:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

raminqaf left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

flinkbot commented May 29, 2026 •

edited

Loading