Skip to content

AST Matcher documentation - #1485

Open
TristonianJones wants to merge 1 commit into
cel-expr:masterfrom
TristonianJones:ast-matcher-docs
Open

TristonianJones wants to merge 1 commit into
cel-expr:masterfrom
TristonianJones:ast-matcher-docs

Conversation

@TristonianJones

Copy link
Copy Markdown
Collaborator

Create documentation for the AST Matcher utilities.

Comment thread docs/ast-matchers.md

| Rule | Example |
| :--- | :--- |
| Map entries match in order | `{'a': _1, 'b': _2}` matches `{'a': 1, 'b': 2}`, not `{'b': 2, 'a': 1}` |

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the ordering requirement there to simplify implementation or is there something structural that prevents us from an unordered match? Same question for struct fields

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a good question. The order matches what appears within the ast.Equiv. Currently, it's pretty strict and requires matching in the exact order of declaration. If we loosen this requirement for maps and structs, then ast.Equiv becomes more powerful and the matcher should inherit the same leniency in match order within these types.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can defer, but just from an expected behavior standpoint, I'd prefer seeing unordered match supported at some point (since maps and struct fields are unordered in CEL). In relation to any key/value matching, it would prevent surprises like:

matcher: {'env': 'prod', _1.star(): _2.star()}

matches: {'env': 'prod', 'region': 'us'}
does not match: {'region': 'us', 'env': 'prod'}

Comment thread docs/ast-matchers.md

> [!NOTE]
> Matching is structural. Proof of semantic equivalence is a separate concern,
> available from the CEL-Java verifier tool.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: consider linking in documentation

Comment thread docs/ast-matchers.md

| Macro | Expansion | Cardinality |
| :--- | :--- | :--- |
| `_1.optional()` | `_1.atMost(1)` | 0 or 1 |

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use something other than optional (or maybe just exclude it)? To avoid confusion and collision with our first class optional type.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have it here mostly for equivalence with regex ?, but I'm okay with just saying atMost.

Comment thread docs/ast-matchers.md
`_1.all(x, x > 0)` matches `items.all(it, it > 0)`.
4. **Map sequences**: entries match in order and each consumes one key and one
value, so `{_1.star(): _2.star()}` matches any map while `{_1: _2.star()}`
matches a single entry.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm having a hard time grokking this section, maybe an explanation might help.

Why does the following need a star? What's the difference between the following two?

// We say this matches a single entry
{_1: _2.star()}

// What would this match differently versus the above?  
{_1: _2}

Also, .star() is supposed to match "0 or more". Does this mean this will also match an empty map? (since a map entry cannot have an empty value).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since _1 always "exactly one", the _2.star() could only ever match a single entry for a valid map-literal. In your example, the behavior would be identical to {_1: _2}. Whereas {_1.star(): _2.star()} would match any shape of map (empty, 1, N). Does that explanation make sense?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that {_1: _2} effectively behaves the same as {_1: _2.star()} (and {_1.star(): _2}, though this case is interesting type-constraint wise on the key), would it be better to stay opinionated and just treat those as an error?. It would connote better that a map must have an atomic entry quantifier (having a star invites cardinality into consideration on either key/value).

I honestly think {_1.star(): _2.star()} is pretty confusing too, but maybe this one's ok if there's no other way of doing a partial map match (empty, 1, N case). Or would _.exprKind(map) work?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

{_1.star(): _2.star()} is probably ok, for doing things like "matching that a specific key exists, followed by any number of optional entries (ex: {'env': 'prod', _1.star(): _2.star()})

@TristonianJones TristonianJones Sep 16, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_1.exprKind(map) will catch the entire map, {_.star(): _1.star()} will catch all values, {_1.star(): _.star()} will catch all keys. Expressions like {_1.atMost(3): _2.atMost(3)} is interesting, {_1.type(string).atMost(3): _2.type(string).atMost(3)} are also interesting. It's not perfect, but not bad considering the behavior is well-defined.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting, I think that would imply the following about the cardinality:

{_1.atMost(2): _2.atMost(5)} -> matches at most 2 entries?
{_1.plus(): _2.star()} -> matches at least 1 entry
{_1: _2.star()} -> `matches exactly 1 entry`

I would think at least the first case might be better served as a compilation error (since there's no real benefit in allowing users in writing a disjoint bound instead of a uniform one).

Comment thread docs/ast-matchers.md

### 7.1 Declarative AST Optimizers & Term Rewriters

Authoring optimizers still requires procedural transformation logic. A

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Food for thought -- the pattern matching currently is agnostic of their types.

My question: should it? There are two reasons why I bring it up for consideration

  1. Things like maps or field selections are type constrained (map keys must be comparable, field selections must be done on strings, with a few exceptions like indexor or backtick accessors). An untyped matching in either case might be too permissive
  2. Thinking ahead for optimization cases -- there is a class of mutations that's unsafe type wise (dyn_x in list, dyn_x && true, associativity rewrite that can lead to precision errors or overflows etc).

@TristonianJones TristonianJones Sep 16, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's possible to use type(<type-name>) to constrain the type of match, e.g. _1.type(string) == null; however, the matcher should always be applied to a type-checked expression so the expectation is that you're matching against something where these constraints already hold. Since the match is only intended to figure out which expr nodes in the expression to extract, there may be cases where the content needs to be inspected post-match.

I think dyn would use the matches everything behavior, but we could make matchers be much more specific: 'only match dyn typed nodes', or _1.notType(dyn) && _2 as the case may be. What do you think?

@l46kok l46kok Sep 16, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

notType does sound useful (dyns are generally where the myriads of edge cases come from wrt/ rewriting). I'd be in favor of it .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants