Skip to content

Compiled Cypher: an aggregating item over a key that is a variable's property is refused, where Neo4j groups by the property (port of drivine4j#491) #311

Closed

Description

@jasperblues

Port of liberation-data/drivine4j#491.

What happens here now. _require_explicit_grouping (drivine/cypher/sql/sql_cypher_compiler.py:6685 on main) takes only a variable as a key inside an item that aggregates (keys at 6691-6695, variables_outside_aggregates at 6698). So these are refused, though Neo4j 2026.10.1 runs them:

UNWIND [{a: 1}, {a: 1}, {a: 2}] AS x RETURN x.a AS a, x.a + count(*) AS b ORDER BY a        -- Neo4j: [1, 3], [2, 3]
MATCH (:Person)-[r:ACTED_IN]->() RETURN r.role AS role, r.role + count(*) AS n ORDER BY role  -- one "Neo" row: "Neo2"
MATCH (p:Person)-[:ACTED_IN]->(m) RETURN p.name, reduce(s = p.name, t IN collect(m.title) | s + t)
MATCH (p:Person)-[:ACTED_IN]->(m) RETURN p.name, p.name IN collect(m.title)

The GROUP BY (sql_cypher_compiler.py:3078) also adds each variable an aggregating item names, so once accepted, r.role + count(*) would be grouped by r too and give a row for each relationship. It must be grouped by the property.

An unnamed map projection is named by its text (drivine/cypher/cypher_reader.py:310-313), where Neo4j names it after its variable. So MATCH (p:Person)-[:ACTED_IN]->(m) RETURN p {.name}, collect(m.title) ORDER BY p.name fails with "p is not defined", and so does WITH p {.name}, count(*) AS c RETURN p.name, c.

Function names in any case. An order term is matched to an item by == (sql_cypher_compiler.py:3028, 6549, 6575), which compares a function's name as written. Neo4j names a function in any case, so UNWIND [{a: 'x'}, {a: 'x'}, {a: 'y'}] AS x RETURN toUpper(x.a), count(*) ORDER BY TOUPPER(x.a) + count(*) is refused here, though Neo4j gives ["X", 2], ["Y", 1]. drivine compares a key with expressionKey, which lowercases function names, in liberation-data/drivine#493. drivine4j matches an order term to an item through keyOf in project.

#310 (in flight) ports drivine4j#482 and changes _require_explicit_grouping. That port reads a key of any shape in a body, which Neo4j doesn't (see below).

What drivine4j's fix does (liberation-data/drivine4j#491 in liberation-data/drivine4j#496, spec liberation-data/drivine-cypher-spec#39):

  • Inside an item that aggregates, beside its aggregates and in a body, a key is read only where it is a variable or a variable's property (isPlainKey). Neo4j refuses (42I18) a key of any other shape there: x.a.b, x['a'], x.a + 1, toUpper(p.name), at the item level and in a body. In a pattern's parts, only a key that is a variable counts: size([(q:Person)-[:KNOWS]->() WHERE q.name = p.name | q]) + count(*) beside p.name is refused, as in Neo4j.
  • aggregatesFirst takes an aggregating item that reads a property key, and works it out over the keys, so it is grouped by the property. A list comprehension's WHERE reads the key by its name. In a comprehension's projection, a quantifier's WHERE or a reduce()'s step, the key's variable stays beside, because Neo4j groups by it there too: [t IN collect(m.title) | r.role] beside r.role gives two "Neo" rows on Neo4j, while size([t IN collect(m.title) WHERE t STARTS WITH r.role | t]) gives one.
  • An unnamed map projection is named after its variable.
  • An order term written as an item, but with its functions named in another case, is that item.

Tests to port: CompilerRefusalTest "an aggregating item reads a key that is a variable or a variable's property, and an order term a key written in any case", and CypherQueryContract "an aggregating item over a key that is a property groups by the property, as Neo4j does". Both have Neo4j's rows. Once the spec is tagged, the new steps of "Implicit grouping key" (queries.yaml) and "Implicit grouping key in a body or an ORDER BY" (refusals.yaml) cover the same cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions