Skip to content

Window sort pushdown below LIMIT/OFFSET prevents early termination #25378

Description

@AdamGS

Describe the bug

Ran into this issue when working on apache/sedona-spatialbench#146, seems like some change since the 54 release changed the behavior here.

To Reproduce

Assuming:

-- Table
CREATE TABLE t (id INT, kind VARCHAR) AS VALUES
  (19, 'a'), (18, 'a'), (17, 'a'), (16, 'a'), (15, 'a'),
  (14, 'a'), (13, 'a'), (12, 'a'), (11, 'a'), (10, 'a'),
  (9, 'a'), (8, 'a'), (7, 'a'), (6, 'a'), (5, 'a'),
  (4, 'a'), (3, 'a'), (2, 'a'), (1, 'a'), (0, 'a');

-- Query
SELECT row_number() OVER (ORDER BY id) AS rn, id
FROM (SELECT id FROM t WHERE kind = 'a' LIMIT 5 OFFSET 5);

The plan we get in an SLT test is:

01)Projection: row_number() ORDER BY [t.id ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW AS rn, t.id
02)--WindowAggr: windowExpr=[[row_number() ORDER BY [t.id ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW]]
03)----Projection: t.id
04)------Limit: skip=5, fetch=5
05)--------Filter: t.kind = Utf8View("a")
06)----------TableScan: t projection=[id, kind]
physical_plan
01)ProjectionExec: expr=[row_number() ORDER BY [t.id ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW@1 as rn, id@0 as id]
02)--BoundedWindowAggExec: wdw=[row_number() ORDER BY [t.id ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW: Field { "row_number() ORDER BY [t.id ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW": UInt64 }, frame: RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW], mode=[Sorted]
03)----ProjectionExec: expr=[id@0 as id]
04)------GlobalLimitExec: skip=5, fetch=5
05)--------SortExec: TopK(fetch=10), expr=[id@0 ASC NULLS LAST], preserve_partitioning=[false]
06)----------FilterExec: kind@1 = a
07)------------DataSourceExec: partitions=1, partition_sizes=[1]

Expected behavior

On 54, the physical plan is:

physical_plan
01)ProjectionExec: expr=[row_number() ORDER BY [t.id ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW@1 as rn, id@0 as id]
02)--BoundedWindowAggExec: wdw=[row_number() ORDER BY [t.id ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW: Field { "row_number() ORDER BY [t.id ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW": UInt64 }, frame: RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW], mode=[Sorted]
03)----SortExec: expr=[id@0 ASC NULLS LAST], preserve_partitioning=[false]
04)------ProjectionExec: expr=[id@0 as id]
05)--------GlobalLimitExec: skip=5, fetch=5
06)----------FilterExec: kind@1 = a, fetch=10
07)------------DataSourceExec: partitions=1, partition_sizes=[1]

Additional context

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions