Refine variant reason computation to include SPLIT and STATIC#11037
Refine variant reason computation to include SPLIT and STATIC#11037
Conversation
|
Hi! 👋 Thanks for your pull request! 🎉 To help us review it, please make sure to:
If you need help, please check our contributing guidelines. |
| .reason( | ||
| !isEmpty(allocation.rules) | ||
| ? Reason.TARGETING_MATCH.name() | ||
| : !isEmpty(split.shards) ? Reason.SPLIT.name() : Reason.STATIC.name()) |
…ng TARGETING_MATCH - Add Split parameter to resolveVariant for three-way reason computation - Return STATIC for flags with no rules and no shards (simple flags) - Return SPLIT for flags with no rules but with shards (shard-based flags) - Return TARGETING_MATCH for flags with rules (rule-based flags) - Update 9 test assertions: 7 to STATIC, 2 to SPLIT
0917bde to
63e4fa3
Compare
BenchmarksStartupParameters
See matching parameters
SummaryFound 1 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 10 unstable metrics.
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~63e4fa3a40, baseline=1.62.0-SNAPSHOT~f064e18a6c
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.06 s) : 0, 1060237
Total [baseline] (11.092 s) : 0, 11091743
Agent [candidate] (1.059 s) : 0, 1059195
Total [candidate] (11.035 s) : 0, 11034506
section appsec
Agent [baseline] (1.252 s) : 0, 1252268
Total [baseline] (11.093 s) : 0, 11092523
Agent [candidate] (1.251 s) : 0, 1251171
Total [candidate] (11.067 s) : 0, 11067041
section iast
Agent [baseline] (1.242 s) : 0, 1241889
Total [baseline] (11.332 s) : 0, 11332180
Agent [candidate] (1.226 s) : 0, 1225751
Total [candidate] (11.233 s) : 0, 11233245
section profiling
Agent [baseline] (1.187 s) : 0, 1187253
Total [baseline] (11.006 s) : 0, 11005768
Agent [candidate] (1.19 s) : 0, 1190377
Total [candidate] (11.132 s) : 0, 11131759
gantt
title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~63e4fa3a40, baseline=1.62.0-SNAPSHOT~f064e18a6c
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.246 ms) : 0, 1246
crashtracking [candidate] (1.224 ms) : 0, 1224
BytebuddyAgent [baseline] (633.543 ms) : 0, 633543
BytebuddyAgent [candidate] (632.543 ms) : 0, 632543
AgentMeter [baseline] (29.563 ms) : 0, 29563
AgentMeter [candidate] (29.389 ms) : 0, 29389
GlobalTracer [baseline] (250.136 ms) : 0, 250136
GlobalTracer [candidate] (249.285 ms) : 0, 249285
AppSec [baseline] (32.554 ms) : 0, 32554
AppSec [candidate] (32.273 ms) : 0, 32273
Debugger [baseline] (60.048 ms) : 0, 60048
Debugger [candidate] (59.878 ms) : 0, 59878
Remote Config [baseline] (596.117 µs) : 0, 596
Remote Config [candidate] (591.088 µs) : 0, 591
Telemetry [baseline] (8.097 ms) : 0, 8097
Telemetry [candidate] (8.051 ms) : 0, 8051
Flare Poller [baseline] (8.314 ms) : 0, 8314
Flare Poller [candidate] (9.961 ms) : 0, 9961
section appsec
crashtracking [baseline] (1.228 ms) : 0, 1228
crashtracking [candidate] (1.227 ms) : 0, 1227
BytebuddyAgent [baseline] (663.463 ms) : 0, 663463
BytebuddyAgent [candidate] (663.472 ms) : 0, 663472
AgentMeter [baseline] (12.078 ms) : 0, 12078
AgentMeter [candidate] (12.138 ms) : 0, 12138
GlobalTracer [baseline] (249.877 ms) : 0, 249877
GlobalTracer [candidate] (250.831 ms) : 0, 250831
IAST [baseline] (24.611 ms) : 0, 24611
IAST [candidate] (24.59 ms) : 0, 24590
AppSec [baseline] (185.544 ms) : 0, 185544
AppSec [candidate] (184.48 ms) : 0, 184480
Debugger [baseline] (66.225 ms) : 0, 66225
Debugger [candidate] (65.692 ms) : 0, 65692
Remote Config [baseline] (638.673 µs) : 0, 639
Remote Config [candidate] (583.915 µs) : 0, 584
Telemetry [baseline] (8.505 ms) : 0, 8505
Telemetry [candidate] (8.356 ms) : 0, 8356
Flare Poller [baseline] (3.57 ms) : 0, 3570
Flare Poller [candidate] (3.506 ms) : 0, 3506
section iast
crashtracking [baseline] (1.253 ms) : 0, 1253
crashtracking [candidate] (1.221 ms) : 0, 1221
BytebuddyAgent [baseline] (813.651 ms) : 0, 813651
BytebuddyAgent [candidate] (802.486 ms) : 0, 802486
AgentMeter [baseline] (11.585 ms) : 0, 11585
AgentMeter [candidate] (11.376 ms) : 0, 11376
GlobalTracer [baseline] (242.119 ms) : 0, 242119
GlobalTracer [candidate] (239.671 ms) : 0, 239671
IAST [baseline] (26.226 ms) : 0, 26226
IAST [candidate] (25.797 ms) : 0, 25797
AppSec [baseline] (32.196 ms) : 0, 32196
AppSec [candidate] (31.904 ms) : 0, 31904
Debugger [baseline] (62.407 ms) : 0, 62407
Debugger [candidate] (58.039 ms) : 0, 58039
Remote Config [baseline] (559.007 µs) : 0, 559
Remote Config [candidate] (1.706 ms) : 0, 1706
Telemetry [baseline] (11.726 ms) : 0, 11726
Telemetry [candidate] (14.086 ms) : 0, 14086
Flare Poller [baseline] (3.582 ms) : 0, 3582
Flare Poller [candidate] (3.406 ms) : 0, 3406
section profiling
crashtracking [baseline] (1.172 ms) : 0, 1172
crashtracking [candidate] (1.195 ms) : 0, 1195
BytebuddyAgent [baseline] (692.654 ms) : 0, 692654
BytebuddyAgent [candidate] (695.802 ms) : 0, 695802
AgentMeter [baseline] (9.148 ms) : 0, 9148
AgentMeter [candidate] (9.179 ms) : 0, 9179
GlobalTracer [baseline] (207.98 ms) : 0, 207980
GlobalTracer [candidate] (208.462 ms) : 0, 208462
AppSec [baseline] (32.86 ms) : 0, 32860
AppSec [candidate] (32.804 ms) : 0, 32804
Debugger [baseline] (66.043 ms) : 0, 66043
Debugger [candidate] (65.854 ms) : 0, 65854
Remote Config [baseline] (588.145 µs) : 0, 588
Remote Config [candidate] (574.182 µs) : 0, 574
Telemetry [baseline] (7.857 ms) : 0, 7857
Telemetry [candidate] (7.865 ms) : 0, 7865
Flare Poller [baseline] (3.564 ms) : 0, 3564
Flare Poller [candidate] (3.562 ms) : 0, 3562
ProfilingAgent [baseline] (93.967 ms) : 0, 93967
ProfilingAgent [candidate] (93.482 ms) : 0, 93482
Profiling [baseline] (94.559 ms) : 0, 94559
Profiling [candidate] (94.057 ms) : 0, 94057
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~63e4fa3a40, baseline=1.62.0-SNAPSHOT~f064e18a6c
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.055 s) : 0, 1054898
Total [baseline] (8.829 s) : 0, 8828798
Agent [candidate] (1.055 s) : 0, 1054532
Total [candidate] (8.82 s) : 0, 8820273
section iast
Agent [baseline] (1.244 s) : 0, 1244474
Total [baseline] (9.575 s) : 0, 9574936
Agent [candidate] (1.232 s) : 0, 1232026
Total [candidate] (9.551 s) : 0, 9551298
gantt
title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~63e4fa3a40, baseline=1.62.0-SNAPSHOT~f064e18a6c
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.22 ms) : 0, 1220
crashtracking [candidate] (1.214 ms) : 0, 1214
BytebuddyAgent [baseline] (632.075 ms) : 0, 632075
BytebuddyAgent [candidate] (631.094 ms) : 0, 631094
AgentMeter [baseline] (29.372 ms) : 0, 29372
AgentMeter [candidate] (29.326 ms) : 0, 29326
GlobalTracer [baseline] (248.626 ms) : 0, 248626
GlobalTracer [candidate] (248.264 ms) : 0, 248264
AppSec [baseline] (32.267 ms) : 0, 32267
AppSec [candidate] (32.153 ms) : 0, 32153
Debugger [baseline] (59.236 ms) : 0, 59236
Debugger [candidate] (58.913 ms) : 0, 58913
Remote Config [baseline] (595.885 µs) : 0, 596
Remote Config [candidate] (583.722 µs) : 0, 584
Telemetry [baseline] (8.027 ms) : 0, 8027
Telemetry [candidate] (8.077 ms) : 0, 8077
Flare Poller [baseline] (7.394 ms) : 0, 7394
Flare Poller [candidate] (8.865 ms) : 0, 8865
section iast
crashtracking [baseline] (1.257 ms) : 0, 1257
crashtracking [candidate] (1.259 ms) : 0, 1259
BytebuddyAgent [baseline] (815.816 ms) : 0, 815816
BytebuddyAgent [candidate] (807.279 ms) : 0, 807279
AgentMeter [baseline] (11.858 ms) : 0, 11858
AgentMeter [candidate] (11.443 ms) : 0, 11443
GlobalTracer [baseline] (242.512 ms) : 0, 242512
GlobalTracer [candidate] (240.863 ms) : 0, 240863
IAST [baseline] (26.252 ms) : 0, 26252
IAST [candidate] (25.961 ms) : 0, 25961
AppSec [baseline] (32.521 ms) : 0, 32521
AppSec [candidate] (29.758 ms) : 0, 29758
Debugger [baseline] (60.396 ms) : 0, 60396
Debugger [candidate] (62.332 ms) : 0, 62332
Remote Config [baseline] (1.725 ms) : 0, 1725
Remote Config [candidate] (529.005 µs) : 0, 529
Telemetry [baseline] (11.686 ms) : 0, 11686
Telemetry [candidate] (12.755 ms) : 0, 12755
Flare Poller [baseline] (3.493 ms) : 0, 3493
Flare Poller [candidate] (3.446 ms) : 0, 3446
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 1 performance regressions! Performance is the same for 19 metrics, 16 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~63e4fa3a40, baseline=1.62.0-SNAPSHOT~f064e18a6c
dateFormat X
axisFormat %s
section baseline
no_agent (18.017 ms) : 17831, 18202
. : milestone, 18017,
appsec (18.882 ms) : 18692, 19072
. : milestone, 18882,
code_origins (17.869 ms) : 17698, 18040
. : milestone, 17869,
iast (17.867 ms) : 17691, 18043
. : milestone, 17867,
profiling (18.52 ms) : 18336, 18704
. : milestone, 18520,
tracing (17.849 ms) : 17672, 18027
. : milestone, 17849,
section candidate
no_agent (18.519 ms) : 18331, 18707
. : milestone, 18519,
appsec (18.691 ms) : 18504, 18878
. : milestone, 18691,
code_origins (17.997 ms) : 17822, 18171
. : milestone, 17997,
iast (18.133 ms) : 17953, 18313
. : milestone, 18133,
profiling (18.807 ms) : 18621, 18992
. : milestone, 18807,
tracing (18.026 ms) : 17848, 18205
. : milestone, 18026,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~63e4fa3a40, baseline=1.62.0-SNAPSHOT~f064e18a6c
dateFormat X
axisFormat %s
section baseline
no_agent (1.236 ms) : 1224, 1247
. : milestone, 1236,
iast (3.279 ms) : 3231, 3328
. : milestone, 3279,
iast_FULL (6.219 ms) : 6155, 6284
. : milestone, 6219,
iast_GLOBAL (3.756 ms) : 3699, 3813
. : milestone, 3756,
profiling (2.149 ms) : 2130, 2169
. : milestone, 2149,
tracing (1.846 ms) : 1830, 1862
. : milestone, 1846,
section candidate
no_agent (1.259 ms) : 1246, 1271
. : milestone, 1259,
iast (3.359 ms) : 3313, 3405
. : milestone, 3359,
iast_FULL (6.163 ms) : 6101, 6226
. : milestone, 6163,
iast_GLOBAL (3.675 ms) : 3614, 3737
. : milestone, 3675,
profiling (2.207 ms) : 2184, 2230
. : milestone, 2207,
tracing (1.884 ms) : 1869, 1899
. : milestone, 1884,
DacapoParameters
See matching parameters
SummaryFound 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 0 unstable metrics.
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~63e4fa3a40, baseline=1.62.0-SNAPSHOT~f064e18a6c
dateFormat X
axisFormat %s
section baseline
no_agent (1.487 ms) : 1476, 1499
. : milestone, 1487,
appsec (3.851 ms) : 3628, 4075
. : milestone, 3851,
iast (2.282 ms) : 2212, 2352
. : milestone, 2282,
iast_GLOBAL (2.321 ms) : 2251, 2392
. : milestone, 2321,
profiling (2.097 ms) : 2042, 2152
. : milestone, 2097,
tracing (2.084 ms) : 2030, 2138
. : milestone, 2084,
section candidate
no_agent (1.49 ms) : 1478, 1501
. : milestone, 1490,
appsec (2.551 ms) : 2496, 2607
. : milestone, 2551,
iast (2.267 ms) : 2197, 2336
. : milestone, 2267,
iast_GLOBAL (2.332 ms) : 2261, 2402
. : milestone, 2332,
profiling (2.115 ms) : 2060, 2170
. : milestone, 2115,
tracing (2.082 ms) : 2028, 2136
. : milestone, 2082,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~63e4fa3a40, baseline=1.62.0-SNAPSHOT~f064e18a6c
dateFormat X
axisFormat %s
section baseline
no_agent (15.038 s) : 15038000, 15038000
. : milestone, 15038000,
appsec (14.896 s) : 14896000, 14896000
. : milestone, 14896000,
iast (18.288 s) : 18288000, 18288000
. : milestone, 18288000,
iast_GLOBAL (17.836 s) : 17836000, 17836000
. : milestone, 17836000,
profiling (14.755 s) : 14755000, 14755000
. : milestone, 14755000,
tracing (14.902 s) : 14902000, 14902000
. : milestone, 14902000,
section candidate
no_agent (14.889 s) : 14889000, 14889000
. : milestone, 14889000,
appsec (14.831 s) : 14831000, 14831000
. : milestone, 14831000,
iast (18.482 s) : 18482000, 18482000
. : milestone, 18482000,
iast_GLOBAL (18.009 s) : 18009000, 18009000
. : milestone, 18009000,
profiling (15.078 s) : 15078000, 15078000
. : milestone, 15078000,
tracing (14.982 s) : 14982000, 14982000
. : milestone, 14982000,
|
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in Use ⏳ Processing |
What Does This Do
Computes the correct OpenFeature evaluation reason from the UFC allocation structure instead of hardcoding
TARGETING_MATCHfor all successful evaluations.Three-way logic in
resolveVariant():STATIC— no rules, no shards (unconditional allocation)SPLIT— no rules, shards present (percentage-based traffic split)TARGETING_MATCH— rules present (flag matched a targeting rule)Also fixes one test assertion that needed updating after rebasing onto #10990: a null targeting key on a static flag now correctly evaluates as
STATICrather thanTARGETING_MATCH.Motivation
System tests
Test_FFE_Eval_Metric_Basic(expectsreason=static) andTest_FFE_Eval_Reason_Split(expectsreason=split) were failing because the evaluator hardcodedTARGETING_MATCHregardless of allocation structure. This aligns Java behavior with the Python and Go SDK reference implementations.Co-authored-by: Leo Romanovsky leo.romanovsky@datadoghq.com (original fix in #10971)
Additional Notes
The
splitobject is passed intoresolveVariant()so the reason can be derived at resolution time without additional lookups.The semantic question of whether a catch-all (no-rules, no-shards) allocation inside a multi-allocation flag should return
STATICvs another reason is out of scope here — that's a pre-existing behavior question tracked separately.Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issueUse
solvesinstead, and assign the PR milestone to the issue