Main - #1228
Open
secrethomefamily-sudo wants to merge 210 commits into
Open
Main#1228secrethomefamily-sudo wants to merge 210 commits into
secrethomefamily-sudo wants to merge 210 commits into
Conversation
Supercedes cel-expr#957 PiperOrigin-RevId: 874711556
…ironment YAML format. PiperOrigin-RevId: 874728744
PiperOrigin-RevId: 874895299
… configs. PiperOrigin-RevId: 875343469
PiperOrigin-RevId: 875410045
PiperOrigin-RevId: 875751638
PiperOrigin-RevId: 875869202
PiperOrigin-RevId: 875912898
PiperOrigin-RevId: 875937520
PiperOrigin-RevId: 876477992
PiperOrigin-RevId: 876500682
PiperOrigin-RevId: 877448588
Updates Environment export to render types in variable and function declarations as structured maps instead of a string of the formatted name. PiperOrigin-RevId: 879147753
google.protobuf.NullValue is represented as an enum (int in CEL), but is interpreted to mean a null literal when set as the alternative in google.protobuf.Value. It is not normally referenced directly, but should behave as an int when it is. PiperOrigin-RevId: 881621568
PiperOrigin-RevId: 882683464
Signed-off-by: Salman Muin Kayser Chishti <13schishti@gmail.com>
…-node24-general PiperOrigin-RevId: 883442166
PiperOrigin-RevId: 885217146
PiperOrigin-RevId: 885225062
PiperOrigin-RevId: 886258398
Includes: - Container setting is made optional to prevent the CEL environment from overriding an already set container with an empty one in case the env is extended multiple times - Brings CEL environment YAML serialization in parity (examples, type, description) - Fixes "comprehensions" to be "two-var-comprehensions" - Partitioned newVaueString into newYamlString and newSourceString, where the former respects YAML multiline syntax PiperOrigin-RevId: 886408709
PiperOrigin-RevId: 886867078
PiperOrigin-RevId: 890106179
…introduce BindingTransformer PiperOrigin-RevId: 890595249
PiperOrigin-RevId: 890628970
Add `strings.quote` and `reverse` extensions to match Go implementations.
PiperOrigin-RevId: 964262762
…block overload declaration PiperOrigin-RevId: 964402265
PiperOrigin-RevId: 964936007
Add more aggregate policy conformance test cases PiperOrigin-RevId: 966209180
PiperOrigin-RevId: 966243304
PiperOrigin-RevId: 966765679
PiperOrigin-RevId: 967407286
PiperOrigin-RevId: 971903461
PiperOrigin-RevId: 973931384
PiperOrigin-RevId: 975968798
PiperOrigin-RevId: 977315425
PiperOrigin-RevId: 978644296
PiperOrigin-RevId: 978680353
…g the AST in `CelOptimizer` if it was not modified. PiperOrigin-RevId: 979038969
…re made. PiperOrigin-RevId: 979113076
…d heterogeneous list literals. PiperOrigin-RevId: 979138470
PiperOrigin-RevId: 979301965
PiperOrigin-RevId: 979344413
…spatcher Split Parser.java into AntlrParser.java (containing all ANTLR dependencies) and Parser.java (acting as a simple dispatcher between AntlrParser and PrattParser). Refactor CelParserParameterizedTest and CelParserImplTest to run tests against both ANTLR and Pratt parsers to ensure exact AST equivalence. PiperOrigin-RevId: 979492678
…I contracts PiperOrigin-RevId: 979522474
PiperOrigin-RevId: 980006129
Improve parsing throughput and reduce memory allocations across CEL expressions. Measured with CelParserBenchmark (parseOnly, built -c opt), comparing three parsers back to back in one session: ANTLR, the Pratt parser before this change, and the Pratt parser after it. Objects allocated per parse: | Case | ANTLR | Pratt before | Pratt after | Pratt vs ANTLR | Delta this CL | | :--- | ---: | ---: | ---: | ---: | ---: | | SMOKE_TEST | 357 | 131 | 123 | 2.9x smaller | -6.1% | | CHAINED_ORS | 968 | 374 | 350 | 2.8x smaller | -6.4% | | LIST_COMPREHENSION | 512 | 218 | 166 | 3.1x smaller | -23.9% | | MESSAGE_CREATION | 1,253 | 502 | 427 | 2.9x smaller | -14.9% | | LONG_LIST | 81,794 | 19,310 | 19,271 | 4.2x smaller | -0.2% | Bytes allocated per parse: | Case | ANTLR | Pratt before | Pratt after | Pratt vs ANTLR | Delta this CL | | :--- | ---: | ---: | ---: | ---: | ---: | | SMOKE_TEST | 12,256 | 3,776 | 3,608 | 3.4x smaller | -4.4% | | CHAINED_ORS | 32,160 | 10,288 | 9,864 | 3.3x smaller | -4.1% | | LIST_COMPREHENSION | 17,320 | 6,120 | 4,736 | 3.7x smaller | -22.6% | | MESSAGE_CREATION | 43,128 | 13,888 | 12,240 | 3.5x smaller | -11.9% | | LONG_LIST | 2,907,488 | 553,160 | 568,080 | 5.1x smaller | +2.7% | Wall clock, mean of 3 caliper trial medians: | Case | ANTLR | Pratt before | Pratt after | Pratt vs ANTLR | Delta this CL | | :--- | ---: | ---: | ---: | ---: | ---: | | SMOKE_TEST | 4,940 ns | 733 ns | 692 ns | 7.1x faster | -5.7% | | CHAINED_ORS | 14,641 ns | 2,090 ns | 2,041 ns | 7.2x faster | -2.3% | | LIST_COMPREHENSION | 7,514 ns | 1,640 ns | 1,218 ns | 6.2x faster | -25.7% | | MESSAGE_CREATION | 20,979 ns | 3,962 ns | 3,487 ns | 6.0x faster | -12.0% | | LONG_LIST | 1,616,631 ns | 144,980 ns | 142,583 ns | 11.3x faster | -1.7% | LONG_LIST is the one case that allocates slightly more than before. It is an extreme outlier (1,000 list elements, ~20k objects per parse) and the +2.7% comes from letting the positions map grow from its default capacity instead of presizing it; presizing cost more on every other case, so the tradeoff is worth it. The map is removed entirely later in this series. PiperOrigin-RevId: 980131070
Lexing and parsing turn code point ranges into strings constantly: once for every identifier, keyword and literal. Both call sites spelled this as slice(i, j).toString(), which allocates an intermediate CelCodePointArray view solely to copy out of it and then discard it. Add CelCodePointArray.substring(i, j), which builds the String straight from the backing array, and implement it in each of the four subclasses. toString() becomes final and delegates to substring(0, size()), so the subclasses lose their near-duplicate toString() overrides. Lexer.consumeIdent and PrattParser.getTokenText call the new method. This removes exactly one 32-byte object per token whose text is materialized. Measured with CelParserBenchmark (parseOnly, built -c opt), comparing three parsers back to back in one session: ANTLR, the Pratt parser before this change, and the Pratt parser after it. Objects allocated per parse: | Case | ANTLR | Pratt before | Pratt after | Pratt vs ANTLR | Delta this CL | | :--- | ---: | ---: | ---: | ---: | ---: | | SMOKE_TEST | 357 | 123 | 120 | 3.0x smaller | -2.4% | | CHAINED_ORS | 968 | 349 | 339 | 2.9x smaller | -2.9% | | LIST_COMPREHENSION | 512 | 166 | 160 | 3.2x smaller | -3.6% | | MESSAGE_CREATION | 1,253 | 426 | 406 | 3.1x smaller | -4.7% | | LONG_LIST | 81,794 | 19,265 | 18,263 | 4.5x smaller | -5.2% | Bytes allocated per parse: | Case | ANTLR | Pratt before | Pratt after | Pratt vs ANTLR | Delta this CL | | :--- | ---: | ---: | ---: | ---: | ---: | | SMOKE_TEST | 12,256 | 3,608 | 3,512 | 3.5x smaller | -2.7% | | CHAINED_ORS | 32,160 | 9,912 | 9,592 | 3.4x smaller | -3.2% | | LIST_COMPREHENSION | 17,320 | 4,928 | 4,736 | 3.7x smaller | -3.9% | | MESSAGE_CREATION | 43,128 | 13,056 | 12,416 | 3.5x smaller | -4.9% | | LONG_LIST | 2,907,488 | 563,952 | 531,888 | 5.5x smaller | -5.7% | Wall clock, mean of 3 caliper trial medians: | Case | ANTLR | Pratt before | Pratt after | Pratt vs ANTLR | Delta this CL | | :--- | ---: | ---: | ---: | ---: | ---: | | SMOKE_TEST | 4,940 ns | 696 ns | 684 ns | 7.2x faster | -1.8% | | CHAINED_ORS | 14,641 ns | 2,008 ns | 2,006 ns | 7.3x faster | -0.1% | | LIST_COMPREHENSION | 7,514 ns | 1,250 ns | 1,217 ns | 6.2x faster | -2.7% | | MESSAGE_CREATION | 20,979 ns | 3,526 ns | 3,514 ns | 6.0x faster | -0.4% | | LONG_LIST | 1,616,631 ns | 140,500 ns | 146,640 ns | 11.0x faster | +4.4% | Wall clock is unchanged within measurement noise. The per-case deltas run from -2.7% to +4.4% and straddle zero, which is what a change that removes 3-5% of allocations and no actual work should look like. The LONG_LIST row reads as a regression, but that trial was noisy (per-trial medians 144.6us, 155.7us, 139.6us, against a much tighter 142.1us, 141.1us, 138.3us before) and its fastest observed parse, 134.6us, is below the 135.7us baseline. The win here is allocation volume and the GC pressure that follows from it. This is the first in a series of parser changes; the wall-clock improvements come later in that series. The ANTLR column is included for scale, and shows why the Pratt parser exists. ANTLR is slow enough on LONG_LIST that the case exceeds caliper's default 5 minute per-trial budget and has to be measured with a raised --time-limit. PiperOrigin-RevId: 981290051
PiperOrigin-RevId: 981393659
…rse path Four related allocation reductions. They land together because the last two are coupled: the parser's new one-shot position map only pays off once the source builder can adopt an immutable map instead of copying it. CelExpr. Every newBuilder() allocated a fresh CelNotSet plus an ExprKind wrapper purely to be overwritten a moment later; those become shared NOT_SET_KIND and NOT_SET_EXPR singletons. CelSelect paid that once per node and CelComprehension five times. addArgs, addElements and addEntries used Arrays.asList(...) followed by forEach(list::add), allocating a list view and a capturing lambda per call, and now use Collections.addAll and Iterables.addAll. The ofCall, ofList, ofStruct and ofMap factories hand an already-immutable collection straight to the value class rather than copying it through the builder's mutable ArrayList. PrattParser. Builds calls, lists, maps and structs through those factories, which also shortens the call sites. The expression position map becomes an int[] indexed by expression id rather than a Map<Long, Integer>; ids are dense and handed out sequentially by nextId, so this drops two boxed objects and a hash insert per node. copyPositionsTo then builds the map once, presized. The source content is cached in a field rather than re-fetched through the accessor, and nextSignificantToken loses a loop over WHITESPACE and COMMENT tokens that could never iterate, since Lexer.lex() consumes those internally and never emits them. CelSource.Builder. positions and macroCalls start as empty immutable maps and are copied into a HashMap only when something actually mutates them. Two builders exist per parse, so this removes up to four hash maps. addPositionsMap and addAllMacroCalls adopt an already-immutable argument outright while the builder is still pristine, which makes build()'s copyOf a no-op and removes the second copy of the parser's position map. extensions is created lazily, and addAllExtensions short-circuits on an empty argument, which matters because toBuilder() always calls it. CelValidationResult. Hoists its issue comparator into a constant and replaces a stream().anyMatch(...) with an indexed loop. Every successful parse paid both. Measured with CelParserBenchmark (parseOnly, built -c opt). "Pratt before" is this CL's parent, so the last column is this CL's own contribution. The ANTLR column is the series baseline and is unaffected by any of these changes. Objects allocated per parse: | Case | ANTLR | Pratt before | Pratt after | Pratt vs ANTLR | Delta this CL | | :--- | ---: | ---: | ---: | ---: | ---: | | SMOKE_TEST | 357 | 120 | 70 | 5.1x smaller | -41.7% | | CHAINED_ORS | 968 | 339 | 201 | 4.8x smaller | -40.7% | | LIST_COMPREHENSION | 512 | 160 | 100 | 5.1x smaller | -37.5% | | MESSAGE_CREATION | 1,253 | 406 | 272 | 4.6x smaller | -33.0% | | LONG_LIST | 81,794 | 18,263 | 13,005 | 6.3x smaller | -28.8% | Bytes allocated per parse: | Case | ANTLR | Pratt before | Pratt after | Pratt vs ANTLR | Delta this CL | | :--- | ---: | ---: | ---: | ---: | ---: | | SMOKE_TEST | 12,256 | 3,512 | 2,040 | 6.0x smaller | -41.9% | | CHAINED_ORS | 32,160 | 9,592 | 5,656 | 5.7x smaller | -41.0% | | LIST_COMPREHENSION | 17,320 | 4,736 | 2,928 | 5.9x smaller | -38.2% | | MESSAGE_CREATION | 43,128 | 12,416 | 8,696 | 5.0x smaller | -30.0% | | LONG_LIST | 2,907,488 | 531,888 | 390,992 | 7.4x smaller | -26.5% | Wall clock, mean of 3 caliper trial medians: | Case | ANTLR | Pratt before | Pratt after | Pratt vs ANTLR | Delta this CL | | :--- | ---: | ---: | ---: | ---: | ---: | | SMOKE_TEST | 4,940 ns | 692 ns | 388 ns | 12.7x faster | -43.9% | | CHAINED_ORS | 14,641 ns | 2,055 ns | 1,111 ns | 13.2x faster | -45.9% | | LIST_COMPREHENSION | 7,514 ns | 1,231 ns | 797 ns | 9.4x faster | -35.3% | | MESSAGE_CREATION | 20,979 ns | 3,583 ns | 2,524 ns | 8.3x faster | -29.6% | | LONG_LIST | 1,616,631 ns | 142,764 ns | 117,823 ns | 13.7x faster | -17.5% | PiperOrigin-RevId: 981440440
PiperOrigin-RevId: 981946267
secrethomefamily-sudo
marked this pull request as ready for review
September 15, 2026 19:29
…ed results PiperOrigin-RevId: 981993302
PiperOrigin-RevId: 982012899
PiperOrigin-RevId: 982070232
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
14 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.