Skip to content

query @@ stored on an EQL match column returns no rows: the reversed @@ overload tests containment backwards #1171

Open

Description

@coderdan

Background

EQL (Encrypt Query Language) is the SQL library this repository installs into a customer's PostgreSQL database. It installs as the eql_v3 schema today, and as eql_v4 from the EQL v4 work in #1161. An encrypted text column with a match index stores a bloom filter (bf): the set of bit positions its 3-grams hash to. A query for a substring is a second, smaller filter. col @@ query means "the column's text contains the query's text". It is answered by array containment: the stored filter must contain every bit of the query filter.

Problem

The wrapper behind @@ is right for one operand order and wrong for the other.

  • stored @@ query is match_term(stored) @> match_term(query). This asks whether the stored filter contains the query's bits, which is correct.
  • query @@ stored is generated with the same body, match_term(a) @> match_term(b), where a is now the query. That asks whether the query contains the stored value, which is backwards.

Checked on a fresh eql_v4 install (Postgres 17):

stored{bf:[1,2,3,4]} @@ query{bf:[1,2]}  -> t
query{bf:[1,2]}  @@ stored{bf:[1,2,3,4]} -> f

Impact: writing the operands the other way round, $1::eql_v4.query_text_match @@ name, is a natural thing to do. It returns no rows, silently. It happens for every domain with a match term: text_match, text_search and text_search_ore, and each of their query twins, in both eql_v3 and eql_v4. All of them come from one generator.

Why nothing catches it: the generated wrappers are rendered by crates/eql-codegen (operator_surface.rs; the body is pinned in generate.rs). Every behavioural test writes the stored column on the left. The EQL v4 PostgreSQL test in #1164 now pins the reversed order as a known gap (query @@ stored finds nothing). That assertion should flip when this is fixed.

Proposal

  1. In the operator-surface generator, render the (query, stored) overload of @@ with its operands swapped: match_term(b) @> match_term(a). Swap the empty-needle guard too: cardinality(match_term(a)) > 0 OR cardinality(match_term(b)) = 0. Keep the wrapper inlinable, so the functional GIN index on match_term(col) still engages. The existing comment on needs_empty_bloom_guard explains why it must not be STRICT.
  2. Add a SQLx test per match-capable domain asserting query @@ stored equals stored @@ query, plus an EXPLAIN showing the index engages for the reversed form.
  3. Flip the known-gap assertion in packages/eql/tests/encryption/tests/v4_types.rs to expect the matching row.
  4. Add a changeset for @cipherstash/eql. This changes query results for anyone already writing the reversed form, from "no rows" to the right rows.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    SDKbugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions