wpe-2.46: Regression in benchmarks JetStream, ARES, speedometer

[asurdej-comcast]

We see a regression in some benchmarks with WPEWebKit 2.46 comparing to WPEWebKit 2.38 on one of our devices.

We are using below benchmarks:
https://browserbench.org/JetStream1.1/
https://browserbench.org/ARES-6/
https://browserbench.org/Speedometer2.0/

Initially we tested Amlogic device and Broadcom that showed similar results in both 2.38 and 2.46, but recently on Mediatek device, there is clear regression in JetStream, ARES and speedometer benchmarks. See screenshots at the bottom.
Because of that I wanted to try JetStream2.2 that shows regression as well.
I reviewed previous results from different devices and even the overall results are similar, certain tests differs between 2.38 and 2.46, especially in JetStream. The results from Mediatek are similar to Aml in 2.46 but Mediatek TV performs much better on 2.38 thus it is regression. Mediatek has better CPU, Cortex A73 vs Aml Cortex A55 so it is expected to have better results.
JSC options are exactly the same between devices, we don't modify the defaults.
It's not trivial to debug this since there are multiple tests that gives different results in different configurations. From JetStream2.2 I selected two tests that results are interesting:
MediateK
"hash-map" - that is almost 3 times worse on 2.46 : 12 vs 35
"crypto" - all crypto group is near 2x worse on 2.46: 55 vs 107
Amlogic:
"hash-map" looks more close 20.00 on 2.46 vs 24.00 on 2.38 - Note that mediatek gives worse result in 2.46 here. The difference is much bigger in JetStream1.1 that uses the same JS code but different loop iterations
"crypto" - 62 on 2.46 and 82 on 2.38 - again better than mediatek in 2.46

Quite similar I see when running JSC binary directly, without webkit at all. Below is from Amlogic only since I don't have Mediatek locally:

root@hisense-v2:/tmp/data/JetStream# ../jsc_2.46 testList.js cli.js          
Starting JetStream2
Running hash-map:
    Startup: 18.587
    Worst Case: 21.322
    Average: 23.602
    Score: 21.070
    Wall time: 0:25.490


First: 18.587
Worst: 21.322
Average: 23.602

Total Score:  21.070 

root@hisense-v2:/tmp/data/JetStream# ../jsc_2.38 testList.js cli.js
Starting JetStream2
Running hash-map:
    Startup: 23.148
    Worst Case: 29.806
    Average: 35.274
    Score: 28.980
    Wall time: 0:17.093


First: 23.148
Worst: 29.806
Average: 35.274

Total Score:  28.980 

Tried to extract hash-map test from JetStream and run it in simple page with different configuration. Interestingly, with JIT disabled it looks like 2.45 is "faster". The result here is the raw time of single iteration of hash-map test:

2.38:
useJIT=false, useDFGJIT=false: 2950ms
useJIT=true, useDFGJIT=false: 920-950ms, 2nd attempt: 900-920ms
useJIT=true, useBaselineJIT=false, useDFGJIT=false: 2950ms
useJIT=true, useBaselineJIT=true, useDFGJIT=true: 137ms

2.46:
useJIT=false, useDFGJIT=false: 2650-2700ms, 2nd attempt: 2750ms
useJIT=true, useDFGJIT=false: 1050-1080ms, 2nd attempt 1060ms
useJIT=true, useBaselineJIT=false, useDFGJIT=false: 2750ms
useJIT=true, useBaselineJIT=true, useDFGJIT=true: 208ms

But again, it's quite hard to compare since results are different based on how you run the test. The same test from jsc (hash-map):

root@hisense-v2:/tmp/data/JetStream# JSC_useDFGJIT=false ../jsc_2.46 testList.js cli.js                                   
Starting JetStream2
Running hash-map:
    Startup: 4.926
    Worst Case: 4.800
    Average: 4.982
    Score: 4.902
    Wall time: 2:00.447


First: 4.926
Worst: 4.800
Average: 4.982

Total Score:  4.902 

root@hisense-v2:/tmp/data/JetStream# JSC_useDFGJIT=false ../jsc_2.38 testList.js cli.js
Starting JetStream2
Running hash-map:
    Startup: 5.574
    Worst Case: 5.451
    Average: 5.696
    Score: 5.573
    Wall time: 1:45.371


First: 5.574
Worst: 5.451
Average: 5.696

Total Score:  5.573 

root@hisense-v2:/tmp/data/JetStream# JSC_useBaselineJIT=false ../jsc_2.38 testList.js cli.js
Starting JetStream2
Running hash-map:
    Startup: 1.786
    Worst Case: 1.744
    Average: 1.786
    Score: 1.772
    Wall time: 5:35.928


First: 1.786
Worst: 1.744
Average: 1.786

Total Score:  1.772 

root@hisense-v2:/tmp/data/JetStream# JSC_useBaselineJIT=false ../jsc_2.46 testList.js cli.js
Starting JetStream2
Running hash-map:

    Startup: 1.777
    Worst Case: 1.739
    Average: 1.791
    Score: 1.769
    Wall time: 5:34.973


First: 1.777
Worst: 1.739
Average: 1.791

Total Score:  1.769 

Will try to collect similar data for Mediatek device, with JIT code generated as well

Initial results:
2.46:

2.38:

2.46:

2.38:

2.46:

2.38:

JetStream2.2:
2.38:

2.46:

Below are results from Amlogic device:
2.38

2.46