[jasperblues]

Ports drivine4j's aggregation refusals, #489 and #482, merged together as liberation-data/drivine4j#490. Also syncs the Cypher spec from v0.12.0 to v0.13.0 (liberation-data/drivine-cypher-spec#38).

Fixes #306
Fixes #307

#306: an aggregate in a WHERE (Neo4j's 42I24)

New check_aggregates in drivine/cypher/sql/aggregates.py, a port of drivine4j's Aggregates. It runs before compiling, beside check_variable_kinds, and reads the statement alone, so a statement that compiled before compiles to the same SQL. It refuses an aggregate in the WHERE of:

• a MATCH or a WITH;
• a list or pattern comprehension, a quantifier, or a quantified path pattern;
• EXISTS { … } or COUNT { … }.

It looks in every clause: each UNION branch, UNWIND, SET, MERGE's ON CREATE and ON MATCH, CREATE's properties, FOREACH, and nested CALLs and subqueries. It doesn't read a subquery's own projection, so all of these still compile:

• COUNT { MATCH … WITH count(m) AS c WHERE c > 1 RETURN c }
• CALL (n) { … WITH count(m) AS c WHERE c > 0 RETURN c }
• WITH count(*) AS c WHERE c > 1

is_aggregate and children are passed in, as in drivine4j, so neither module imports the other.

#307: an implicit grouping key in a body or an ORDER BY (Neo4j's 42I18)

_require_explicit_grouping(items, order_by, scope) keeps its old check. It now also reads:

• an aggregating item's comprehension, quantifier, reduce() and pattern bodies (_named_in_bodies);
• after items that aggregate, each ORDER BY term that aggregates (_named_outside).

It matches whole key expressions, not just variables:

• RETURN x, [y IN collect(x) | y + x] compiles, because x is a key.
• RETURN p.name, size([x IN collect(r) WHERE x.role <> p.name]) compiles.
• RETURN x.a AS a, count(*) ORDER BY x.a + count(*) compiles.

After a rename (RETURN p AS q, count(*) …):

• the map projection p {.born}.born reads the key, and compiles;
• p inside a pattern (size([(p)-->(k) | k]), EXISTS { (p)-->() }) is refused. Neo4j reads that p as a new variable that matches any node, and the spec refuses it on SQL too.
• Without the rename, the pattern form compiles.

Tests

• tests/unit/test_cypher_refusals.py gets drivine4j's two CompilerRefusalTest cases on Postgres and Oracle. One has 21 refusals and 6 statements that must compile; the other has 17 refusals and 16 that must compile. Before the fix, all 76 refusal tests (38 × 2 stores) failed, and the 44 compile tests passed.
• tests/integration/test_cypher_compiled.py gets drivine4j's CypherQueryContract case "an aggregate beside a WHERE, a grouping key's body or an order term that Neo4j takes gives Neo4j's rows". scripts/unasync.py copies it to the sync tests.
• Spec v0.13.0 adds "Aggregate in a WHERE" and "Implicit grouping key in a body or an ORDER BY" to refusals.yaml. In queries.yaml it adds "Aggregate beside a WHERE" and new steps of "Implicit grouping key". All pass on Neo4j, Postgres and Oracle.
• Gate: ruff check, ruff format --check, pyright, scripts/unasync.py --check, and the full pytest with ORACLE_TESTS=true: 5840 passed, 298 skipped.

SQL comparison

I compiled 4,179 statements on Postgres and on Oracle, before (main at 1b3ffe7) and after. The statements were:

• every spec statement at v0.6.19, v0.10.0, v0.12.0 and v0.13.0;
• the golden statements;
• every Cypher string in this repo's tests, samples and docs;
• drivine#482's corpus;
• every Cypher literal in drivine4j's CompilerRefusalTest.kt.

SQL, prelude and parameters are compared. No statement that compiled before gives different SQL. 42 distinct statements changed, on both stores:

• 24 were compiled and are now refused for an aggregate in a WHERE (42I24). One more was refused before and gets the new message.
• 9 were compiled and are now refused for an aggregating item whose body names a variable that isn't a key (42I18).
• 7 were compiled and are now refused for an aggregating ORDER BY term (42I18). Two more were refused before with "z is not defined" or "m is not defined", and now get the ORDER BY message.

Each is a statement from refusals.yaml at v0.13.0 or from drivine4j's refusal lists, which Neo4j refuses. There are three exceptions, all forms of RETURN p AS q, count(*) … ORDER BY with p in a pattern. Neo4j runs those, reading p as a new variable. drivine4j and the spec's only: [sql] step refuse them on purpose. tests/golden/statements.json is unchanged, and all 164 golden compilations (82 statements × 2 stores) are the same before and after.

Differences from drivine4j

• The ported unit tests run on Postgres and Oracle only, since there's no SQLite compiler here.
• Variables found are kept as ordered lists, not Kotlin's ordered sets, and key expressions as a list compared by equality. A message names the variables in the same order.

Cross-library: drivine4j has the fix. Drivine for TypeScript tracks it as liberation-data/drivine#487 and #488, which are being ported now.