Tracking: implement ADR-0009, operator-typed query terms for Stack Encrypt #1178

Background

Stack Encrypt produces query terms that a caller binds into SQL through sqlx,
Diesel or (later, through the bindings) Drizzle and Prisma. It never runs a
query. ADR-0009 (#1177, proposed) records the query-side model for the Rust
library: what a caller asks for, how the library picks the index that answers
it, what it hands back, and how far that answer can be trusted. This issue
tracks implementing it.

Scope is the Rust Stack Encrypt library. The bindings and language SDKs follow
in #1188 and #1189.

Problem

• A query term is not the stored term. ADR-0004 decision 5 assumed a probe
  and a stored term are the same bytes. That holds for HMAC only. ORE queries
  with the left ciphertext alone, Bloom match queries with a filter over the
  needle's n-grams, and JSON path queries with a selector. The review of feat(eql-bindings): produce EQL v4 TextMatch from Stack Encrypt match terms #1164
  ran into this: storing short TextMatch text needs the stored and query
  terms to differ, and the API had nowhere to say so.
• The caller names an index, not an operator.
  cipher.query(value).using(&plan).index(Match(..)) leaves the caller to
  choose between indexes, does not reject an operator the index cannot answer
  at compile time, and has no place for operator-specific input (Between's
  two bounds, a JSON path, a fetch size).
• Approximate answers are never signalled. Bloom match, lossy text order
  and similarity all return false positives. A caller who wants ten real
  matches cannot tell how many rows to fetch, or which decrypted rows to keep.

Proposal

ADR-0009's decisions are delivered by these sub-issues, each sized to one PR:

• stack-encrypt: name the operator, not the index, in a query (Eq and Matches) #1182: the operator-typed query, Eq and Matches (decisions 1 to 4).
• stack-encrypt: verify a Matches query's rows after decrypt (Probabilistic) #1183: Probabilistic and verify for Matches (decisions 4 and 5).
• stack-encrypt: size a probabilistic query's fetch with target_limit and Stats #1184: Stats, Limit, target_limit and the expected_tokens prior (decision 5).
• stack-encrypt: store short TextMatch text as an empty filter, refuse only short queries #1185: store short TextMatch text as an empty filter, refuse only short queries.
• stack-encrypt: ordering operators Lt, Lte, Gt, Gte and Between, and remove QueryUsing::index #1186: the ordering operators and Between, and removing QueryUsing::index (decision 7). Blocked by stack-encrypt: delete dynamic::Value and Scalar, derive terms through vitaminc, add block ORE, drop Bool order terms #1140.
• stack-encrypt: JSON operators Contains, ContainedBy and AtPath #1187: the JSON operators Contains, ContainedBy and AtPath. Blocked on a JSON index.
• Mark ADR-0009 accepted once stack-encrypt: size a probabilistic query's fetch with target_limit and Stats #1184 pins Stats and Limit.

The EQL side of decision 8, where the v4 types implement Answers and serialise a query term to its eql_v4.query_* domain, waits for the v4 types to be Stack Encrypt targets (#1179).

Still deferred in the ADR: the Nearest entry point and whether a ranked query takes further predicates, ORDER BY fill on the EQL side, how a Bloom Nearest is scored, and packaging the loops once Stack Similarity exists. The Between spelling and expected_tokens are now decided.

Related

• docs(adr): ADR-0009, operators select the index and a query term is not the stored term #1177: ADR-0009.
• Tracking: implement ADR-0002, new value kinds and EQL v4 for Stack Encrypt #1145: tracks ADR-0002 (the value model) and EQL v4, which this builds on.
• feat(eql-bindings): produce EQL v4 TextMatch from Stack Encrypt match terms #1164: the short TextMatch threads that need the stored/query term split.