[3.x] Speed up compiling integer literals

[andreybolonin]

Compiler::repr() currently queries LC_NUMERIC, switches it to C, and restores it for every integer as well as every float. Integer representations do not need locale switching. Handle integers in a separate branch and retain the existing locale and serialize_precision handling for floats.

Keep using var_export() for integers. In particular, its special representation of PHP_INT_MIN must remain intact so evaluating the generated PHP produces an integer without losing precision. The now float-only branch also no longer needs to check is_float() again before setting serialize_precision.

Measurements

Benchmarked the proposed branch before making the repository change, then repeated the measurements against the exact final patch. The figures below are from the final patch, compared with 428a16179.

Environment: PHP 8.5.11 CLI, macOS arm64, JIT disabled, tested with CLI OPcache both disabled and enabled. Each figure is the median of 9 batches, alternating before/after execution order, after 15 warm-up calls per version. Both complete compiler implementations are loaded as separate subclasses with the same inheritance structure in one process, so the candidate is not measured through an extra overriding wrapper. Times are microseconds per call, not batch totals.

Isolated repr() calls, including reset() before each call:

Input	Calls/version/batch	OPcache off: before → after, µs	Elapsed time change	OPcache on: before → after, µs	Elapsed time change
Integer 123, C locale	100,000	0.371 → 0.191	-48.49%	0.351 → 0.185	-47.27%
range(0, 99), C locale	2,000	81.586 → 45.153	-44.66%	79.594 → 46.477	-41.61%
Float 0.1, C locale	100,000	0.492 → 0.484	-1.61%	0.469 → 0.468	-0.09%
String hello, C locale	100,000	0.176 → 0.171	-3.18%	0.167 → 0.168	+0.79%
Integer 123, French locale	100,000	0.394 → 0.188	-52.17%	0.377 → 0.182	-51.59%

Negative percentages mean less elapsed time. The 100-element array exercises integer keys as well as integer values. Float and string rows are controls; small differences on those paths should not be read as a separate performance claim.

Full Environment::compileSource(), including lexing, parsing, node visitors, and code generation, with a reused environment and warmed compiler/parser/lexer:

{% set x = [1, 2, 3, 4, 5] %}{{ x[0] }}

Repeat this 40-byte line (including its trailing newline) 1, 10, 100, or 1,000 times. Each line has six integer literals in the Twig source. The zero-assignment control is the 13-byte static template Hello, world!.

CLI OPcache disabled:

Assignments	Source size, bytes	Calls/version/batch	Before, µs	After, µs	Change in elapsed time
0	13	500	38.619	38.440	-0.46%
1	40	500	138.950	136.481	-1.78%
10	400	150	872.724	859.531	-1.51%
100	4,000	30	8103.900	8007.279	-1.19%
1,000	40,000	4	87603.125	87076.750	-0.60%

CLI OPcache enabled:

Assignments	Source size, bytes	Calls/version/batch	Before, µs	After, µs	Change in elapsed time
0	13	500	37.688	37.480	-0.55%
1	40	500	131.144	129.676	-1.12%
10	400	150	826.935	815.772	-1.35%
100	4,000	30	7681.629	7592.396	-1.16%
1,000	40,000	4	82644.396	81228.938	-1.71%

The isolated integer path consistently takes about half the time. Full compilation benefits are much smaller because lexing, parsing, and traversal dominate; larger-template results are also more sensitive to run-to-run variation. These measurements concern compilation of the specified numeric workload, not rendering an already compiled/cached template or a universal 2% improvement for all templates. Complete batch ranges are included below.

Behavior and validation

• Identical generated representations for 19 values (integer boundaries, floats including negative zero/INF/NAN, booleans, null, strings, and nested arrays), across C and French numeric locales and serialize_precision values −1, 0, and 17. Locale and precision are preserved after each call.
• Identical generated PHP for each full-compilation workload above.
• Added integer round-trip coverage including PHP_INT_MIN, PHP_INT_MIN + 1, and PHP_INT_MAX; extended locale/precision tests to cover integers alongside floats.
• vendor/bin/simple-phpunit: 2,872 tests, 7,026 assertions, 4 skipped, no failures (99 existing legacy deprecation notices reported).
• vendor/bin/phpstan analyse --no-progress: no errors.
• vendor/bin/php-cs-fixer fix --config=.php-cs-fixer.dist.php --dry-run --diff src/Compiler.php tests/CompilerTest.php: no changes needed.

Raw benchmark results, including batch ranges

CLI OPcache disabled:

PHP 8.5.11; arch arm64; CLI OPcache=0; JIT=disable
Behavior checks: identical output and restored locale/precision for 19 values, 2 locales, 3 precision settings
repr+reset: integer, locale C | n=100000 | 0.371 -> 0.191 us/op | -48.49% elapsed | range 0.368..0.375 / 0.189..0.192
repr+reset: float, locale C | n=100000 | 0.492 -> 0.484 us/op | -1.61% elapsed | range 0.489..0.495 / 0.479..0.522
repr+reset: string, locale C | n=100000 | 0.176 -> 0.171 us/op | -3.18% elapsed | range 0.175..0.184 / 0.168..0.172
repr+reset: integer array, locale C | n=2000 | 81.586 -> 45.153 us/op | -44.66% elapsed | range 80.769..87.634 / 44.644..46.108
compileSource: 0 assignments, 13 bytes | n=500 | 38.619 -> 38.440 us/op | -0.46% elapsed | range 38.333..39.122 / 38.294..38.918
compileSource: 1 assignments, 40 bytes | n=500 | 138.950 -> 136.481 us/op | -1.78% elapsed | range 138.331..140.883 / 136.030..146.977
compileSource: 10 assignments, 400 bytes | n=150 | 872.724 -> 859.531 us/op | -1.51% elapsed | range 870.028..898.488 / 857.383..891.951
compileSource: 100 assignments, 4000 bytes | n=30 | 8103.900 -> 8007.279 us/op | -1.19% elapsed | range 8097.300..8327.257 / 7950.260..8078.982
compileSource: 1000 assignments, 40000 bytes | n=4 | 87603.125 -> 87076.750 us/op | -0.60% elapsed | range 87126.302..89321.812 / 85552.344..88065.052
repr+reset: integer, locale fr_FR.UTF-8 | n=100000 | 0,394 -> 0,188 us/op | -52,17% elapsed | range 0,392..0,397 / 0,184..0,189
compileSource: 100 assignments, locale fr_FR.UTF-8 | n=30 | 8182,725 -> 8026,889 us/op | -1,90% elapsed | range 8142,711..8440,411 / 7997,874..8076,054

CLI OPcache enabled:

PHP 8.5.11; arch arm64; CLI OPcache=1; JIT=disable
Behavior checks: identical output and restored locale/precision for 19 values, 2 locales, 3 precision settings
repr+reset: integer, locale C | n=100000 | 0.351 -> 0.185 us/op | -47.27% elapsed | range 0.348..0.359 / 0.184..0.186
repr+reset: float, locale C | n=100000 | 0.469 -> 0.468 us/op | -0.09% elapsed | range 0.466..0.475 / 0.463..0.470
repr+reset: string, locale C | n=100000 | 0.167 -> 0.168 us/op | +0.79% elapsed | range 0.166..0.169 / 0.166..0.170
repr+reset: integer array, locale C | n=2000 | 79.594 -> 46.477 us/op | -41.61% elapsed | range 79.045..79.935 / 46.228..46.622
compileSource: 0 assignments, 13 bytes | n=500 | 37.688 -> 37.480 us/op | -0.55% elapsed | range 37.222..38.136 / 37.142..37.845
compileSource: 1 assignments, 40 bytes | n=500 | 131.144 -> 129.676 us/op | -1.12% elapsed | range 130.687..131.559 / 129.302..130.230
compileSource: 10 assignments, 400 bytes | n=150 | 826.935 -> 815.772 us/op | -1.35% elapsed | range 822.737..834.198 / 811.776..860.041
compileSource: 100 assignments, 4000 bytes | n=30 | 7681.629 -> 7592.396 us/op | -1.16% elapsed | range 7671.250..7817.719 / 7551.068..7709.415
compileSource: 1000 assignments, 40000 bytes | n=4 | 82644.396 -> 81228.938 us/op | -1.71% elapsed | range 82120.792..84482.948 / 80787.406..84175.073
repr+reset: integer, locale fr_FR.UTF-8 | n=100000 | 0,377 -> 0,182 us/op | -51,59% elapsed | range 0,374..0,387 / 0,181..0,184
compileSource: 100 assignments, locale fr_FR.UTF-8 | n=30 | 7803,431 -> 7678,001 us/op | -1,61% elapsed | range 7722,576..7975,194 / 7567,975..8027,435

Reproduce the benchmark and output/state comparisons

Check out this branch and install Composer dependencies. Save the PHP script below as /tmp/twig-repr-benchmark.php, then run from the repository root:

git show 428a16179:src/Compiler.php > /tmp/twig-compiler-before.php
php -d opcache.enable_cli=0 /tmp/twig-repr-benchmark.php
php -d opcache.enable_cli=1 -d opcache.file_update_protection=0 /tmp/twig-repr-benchmark.php

Both class variants are loaded from files. Disabling OPcache's file update protection for the second command lets it optimize both newly generated variant files equally. Both C and fr_FR.UTF-8 numeric locales are needed for the state comparisons and French-locale measurements. Measurements should be run without other CPU-intensive work.

<?php
require getcwd().'/vendor/autoload.php';
use Twig\Environment;
use Twig\Loader\ArrayLoader;
use Twig\Source;
$beforeCode = file_get_contents('/tmp/twig-compiler-before.php');
$afterCode = file_get_contents(getcwd().'/src/Compiler.php');
foreach (['Before' => $beforeCode, 'After' => $afterCode] as $side => $variant) {
    $file = "/tmp/twig-repr-$side.php";
    file_put_contents($file, str_replace('class Compiler', "class {$side}Compiler extends Compiler", $variant));
    require $file;
}
function compare(string $label, callable $before, callable $after, int $n): void {
    for ($i = 0; $i < 15; ++$i) { $before(); $after(); }
    $times = [[], []];
    for ($r = 0; $r < 9; ++$r) {
        foreach ($r % 2 ? [1, 0] : [0, 1] as $side) {
            $callback = [$before, $after][$side];
            $start = hrtime(true);
            for ($i = 0; $i < $n; ++$i) { $callback(); }
            $times[$side][] = (hrtime(true) - $start) / $n / 1000;
        }
    }
    sort($times[0]); sort($times[1]);
    $a = $times[0][4]; $b = $times[1][4];
    printf("%s | n=%d | %.3f -> %.3f us/op | %+.2f%% elapsed | range %.3f..%.3f / %.3f..%.3f\n", $label, $n, $a, $b, 100 * ($b / $a - 1), $times[0][0], $times[0][8], $times[1][0], $times[1][8]);
}
$envA = new Environment(new ArrayLoader()); $envB = new Environment(new ArrayLoader());
$before = new Twig\BeforeCompiler($envA); $after = new Twig\AfterCompiler($envB);
$envA->setCompiler($before); $envB->setCompiler($after);
printf("PHP %s; arch %s; CLI OPcache=%s; JIT=%s\n", PHP_VERSION, php_uname('m'), ini_get('opcache.enable_cli'), ini_get('opcache.jit'));
$initialLocale = setlocale(LC_NUMERIC, '0'); $initialPrecision = ini_get('serialize_precision');
$values = [PHP_INT_MIN, PHP_INT_MIN + 1, PHP_INT_MAX, -123, -1, 0, 1, 123, 0.1, -0.0, 1e30, INF, -INF, NAN, true, false, null, 'text', [0,1,-1,PHP_INT_MIN,'key'=>[0.1,PHP_INT_MAX]]];
foreach (['C', 'fr_FR.UTF-8'] as $locale) {
    if (false === setlocale(LC_NUMERIC, $locale)) { throw new RuntimeException("Missing locale: $locale"); }
    foreach (['-1', '0', '17'] as $precision) {
        ini_set('serialize_precision', $precision);
        foreach ($values as $value) {
            $expected = $before->reset()->repr($value)->getSource();
            $actual = $after->reset()->repr($value)->getSource();
            if ($expected !== $actual || setlocale(LC_NUMERIC, '0') !== $locale || ini_get('serialize_precision') !== $precision) { throw new RuntimeException('Behavior mismatch'); }
        }
    }
}
setlocale(LC_NUMERIC, 'C'); ini_set('serialize_precision', '-1');
echo "Behavior checks: identical output and restored locale/precision for 19 values, 2 locales, 3 precision settings\n";
foreach (['integer'=>123, 'float'=>0.1, 'string'=>'hello', 'integer array'=>range(0,99)] as $label=>$value) {
    compare("repr+reset: $label, locale C", fn()=>$before->reset()->repr($value), fn()=>$after->reset()->repr($value), is_array($value) ? 2000 : 100000);
}
foreach ([0,1,10,100,1000] as $count) {
    $template = $count ? str_repeat('{% set x = [1, 2, 3, 4, 5] %}{{ x[0] }}' . "\n", $count) : 'Hello, world!';
    $source = new Source($template, 'bench');
    $envA->getLoader()->setTemplate('bench', $template); $envB->getLoader()->setTemplate('bench', $template);
    if ($envA->compileSource($source) !== $envB->compileSource($source)) { throw new RuntimeException('Compiled PHP mismatch'); }
    compare("compileSource: $count assignments, ".strlen($template).' bytes', fn()=>$envA->compileSource($source), fn()=>$envB->compileSource($source), [0=>500,1=>500,10=>150,100=>30,1000=>4][$count]);
}
setlocale(LC_NUMERIC, 'fr_FR.UTF-8');
compare('repr+reset: integer, locale fr_FR.UTF-8', fn()=>$before->reset()->repr(123), fn()=>$after->reset()->repr(123), 100000);
$template = str_repeat('{% set x = [1, 2, 3, 4, 5] %}{{ x[0] }}' . "\n", 100);
$source = new Source($template, 'bench');
$envA->getLoader()->setTemplate('bench', $template); $envB->getLoader()->setTemplate('bench', $template);
compare('compileSource: 100 assignments, locale fr_FR.UTF-8', fn()=>$envA->compileSource($source), fn()=>$envB->compileSource($source), 30);
setlocale(LC_NUMERIC, $initialLocale); ini_set('serialize_precision', $initialPrecision);

Memory measurements

Compared 428a16179 with this MR's exact head in separate, fresh PHP processes (not two renamed classes loaded together). PHP 8.5.11 CLI on macOS arm64, JIT disabled, with CLI OPcache both off and on. Five independent processes per version/workload/OPcache mode, alternating execution order; values below are medians.

Inputs, the environment, and extension initialization are set up before the measured interval. memory_reset_peak_usage() resets the peak immediately before the operation. Reported used-heap peaks come from memory_get_peak_usage(false); reserved peaks come from memory_get_peak_usage(true). Absolute peaks include the existing baseline heap; “incremental peak” subtracts the heap immediately before the operation. Result objects/strings remain alive when memory is sampled. Output verification happens afterward, so its serialization buffers are excluded. Process max RSS includes startup/setup and is sampled before verification. OPcache shared usage is reported separately because it is not part of the PHP used-heap metric.

All before/after output hashes matched. These are memory measurements only; process startup is excluded from the earlier time benchmarks.

Full compileSource(), using the same 40-byte numeric-assignment line as in the time benchmark; the zero-assignment control is Hello, world!:

Assignments	Source size, bytes	Used-heap peak, KiB, OPcache off: before → after	Used-heap peak, KiB, OPcache on: before → after	Reserved peak, MiB, off / on (both versions)
0	13	2,613.156 → 2,617.156	1,367.438 → 1,367.438	4 / 2
1	40	3,103.828 → 3,107.828	1,367.516 → 1,367.516	4 / 2
10	400	3,306.070 → 3,310.070	1,507.359 → 1,507.359	4 / 2
100	4,000	5,346.711 → 5,350.711	3,548.000 → 3,548.000	6 / 4
1,000	40,000	25,533.125 → 25,537.125	23,645.438 → 23,645.438	26 / 24

The incremental peak of compilation is identical before/after at every size in both OPcache modes. Without OPcache, the starting PHP heap is 4,096 B higher after the change, so absolute compilation peaks are also 4,096 B higher. An independent check loading only the Compiler class measured 24,688 → 28,784 B of used PHP heap for the class definition, confirming this is the fixed cost of the modified class rather than per-template allocation. With OPcache enabled, the starting PHP heap and absolute compilation peaks are identical; OPcache-reported shared used memory is 96 B higher after the change.

repr() alone, including reset(), with the integer input/array constructed before resetting the peak:

Input	Incremental peak before, bytes	Incremental peak after, bytes	Retained output heap, bytes (both versions)
Integer 123	320	288	32
Array of 100 integers	1,344	1,312	1,024
Array of 10,000 integers	139,584	139,552	139,264
Array of 100,000 integers	1,581,376	1,581,344	1,581,056

The incremental peaks in this table are the same with OPcache off/on. The integer path uses 32 B less temporary heap, with unchanged retained output memory. This does not constitute a meaningful memory-saving claim.

After a warm-up followed by 100 compilations of a 4,000-byte numeric template, retained heap growth after garbage collection is 0 B in both versions, and the incremental peak is identical. Peak reserved PHP memory is unchanged in every tested workload.

Process max RSS for the largest full-operation workload (includes startup/setup; five fresh processes per side):

OPcache	Before median, MiB (range)	After median, MiB (range)
off	51.094 (51.031–51.141)	51.031 (51.031–51.125)
on	54.406 (54.328–54.438)	54.406 (54.344–54.422)

RSS has page-level/process-startup variation and should not be used to attribute byte-sized differences to this patch; the allocator/retained-heap figures above show the allocation differences directly.

Reproduce the memory benchmark

Install Composer dependencies, save the PHP and Python scripts below as /tmp/twig-memory-probe.php and /tmp/twig-memory-run.py, then run from the repository root:

git show 428a16179:src/Lexer.php > /tmp/twig-memory-lexer-before.php
git show 8df9ea9e2:src/Lexer.php > /tmp/twig-memory-lexer-after.php
git show 428a16179:src/Compiler.php > /tmp/twig-memory-compiler-before.php
git show 8ba033841:src/Compiler.php > /tmp/twig-memory-compiler-after.php
python3 /tmp/twig-memory-run.py

The runner launches 420 fresh PHP processes in total across both MRs. It loads the original class name from the selected snapshot before the Composer autoloader can load it, so each process contains exactly one implementation of the measured class. All other source files must match the MR base; the time/memory changes in these two MRs are confined to the selected Compiler/Lexer methods. Inputs and dependencies outside those classes are shared identically across each comparison. PHP 8.2+ is needed for memory_reset_peak_usage(); the reported results use PHP 8.5.11. The scripts save all individual measurements to /tmp/twig-memory-results.json and print per-case medians.

<?php
require getcwd().'/vendor/autoload.php';
[$script, $target, $side, $scenario, $count] = $argv;
$count = (int) $count;
require "/tmp/twig-memory-$target-$side.php";
$env = new Twig\Environment(new Twig\Loader\ArrayLoader());
// Keep fixture construction and extension initialization outside the measured interval.
$parsers = $env->getExpressionParsers();
if ($target === 'lexer') {
    if ($scenario === 'operators' || $scenario === 'extended') {
        $extra = [];
        for ($i = 0; $i < $count; ++$i) {
            $extra[] = new Twig\ExpressionParser\Prefix\UnaryOperatorExpressionParser(Twig\Node\Expression\Unary\NotUnary::class, 'extra_'.$i, 50, aliases: ['alias_'.$i]);
        }
        $parsers->add($extra);
        unset($extra);
    }
    $unit = "<li>{{ item.name|upper }}: {% if item.price >= 10 and item.active %}{{ item.price * 2 }}{% else %}n/a{% endif %}</li>\n";
    $source = new Twig\Source(str_repeat($unit, $scenario === 'tokenize' ? $count : ($scenario === 'repeat' ? 100 : 0)), 'bench');
    $lexer = new Twig\Lexer($env);
    $method = new ReflectionMethod(Twig\Lexer::class, 'getOperatorRegex');
    $operation = $scenario === 'operators' ? fn() => $method->invoke($lexer) : fn() => $lexer->tokenize($source);
    if ($scenario === 'repeat') {
        $lexer->tokenize($source);
    }
} else {
    $compiler = new Twig\Compiler($env);
    $env->setCompiler($compiler);
    $unit = '{% set x = [1, 2, 3, 4, 5] %}{{ x[0] }}' . "\n";
    $template = $scenario === 'compile' ? ($count ? str_repeat($unit, $count) : 'Hello, world!') : str_repeat($unit, 100);
    $source = new Twig\Source($template, 'bench');
    $env->getLoader()->setTemplate('bench', $template);
    if ($scenario === 'repr') {
        $value = $count ? range(0, $count - 1) : 123;
        $operation = fn() => $compiler->reset()->repr($value)->getSource();
    } else {
        $operation = fn() => $env->compileSource($source);
        if ($scenario === 'repeat') {
            $operation();
        }
    }
}
gc_collect_cycles();
$usedBefore = memory_get_usage(false);
$reservedBefore = memory_get_usage(true);
memory_reset_peak_usage();
$iterations = $scenario === 'repeat' ? 100 : 1;
for ($i = 0; $i < $iterations; ++$i) {
    unset($result);
    $result = $operation();
}
// Capture before hashing/serializing output so verification buffers do not affect the peak.
$usedAfter = memory_get_usage(false);
$peakUsed = memory_get_peak_usage(false);
$reservedAfter = memory_get_usage(true);
$peakReserved = memory_get_peak_usage(true);
gc_collect_cycles();
$usedAfterGc = memory_get_usage(false);
// Capture RSS before output verification allocates serialization buffers.
$rss = getrusage()['ru_maxrss'];
if (PHP_OS_FAMILY === 'Linux') {
    $rss *= 1024;
}
$opcacheStatus = opcache_get_status(false);
$opcacheUsed = $opcacheStatus ? $opcacheStatus['memory_usage']['used_memory'] : null;
if (is_string($result)) {
    $digest = hash('sha256', $result);
} else {
    $digest = hash('sha256', serialize($result));
}
echo json_encode([
    'target'=>$target, 'side'=>$side, 'scenario'=>$scenario, 'count'=>$count,
    'opcache'=>(int) ini_get('opcache.enable_cli'),
    'used_before'=>$usedBefore, 'used_after'=>$usedAfter, 'used_after_gc'=>$usedAfterGc,
    'peak_used'=>$peakUsed, 'peak_delta'=>$peakUsed-$usedBefore,
    'retained_delta'=>$usedAfterGc-$usedBefore,
    'reserved_before'=>$reservedBefore, 'reserved_after'=>$reservedAfter, 'peak_reserved'=>$peakReserved,
    'max_rss'=>$rss, 'opcache_used'=>$opcacheUsed, 'digest'=>$digest,
]), "\n";

import json, subprocess, statistics, pathlib
cases = [('lexer','operators',n) for n in [0,100,1000]]
cases += [('lexer','tokenize',n) for n in [0,1,10,100,1000]]
cases += [('lexer','extended',n) for n in [100,1000]]
cases += [('lexer','repeat',0)]
cases += [('compiler','repr',n) for n in [0,100,10000,100000]]
cases += [('compiler','compile',n) for n in [0,1,10,100,1000]]
cases += [('compiler','repeat',0)]
all_results=[]
for opcache in [0,1]:
    for target,scenario,count in cases:
        pair={}
        for repeat in range(5):
            for side in (['before','after'] if repeat%2==0 else ['after','before']):
                cmd=['php',f'-d',f'opcache.enable_cli={opcache}','-d','opcache.file_update_protection=0','-d','opcache.jit=disable','/tmp/twig-memory-probe.php',target,side,scenario,str(count)]
                result=json.loads(subprocess.check_output(cmd,text=True))
                all_results.append(result)
                pair.setdefault(side,[]).append(result)
        if {v['digest'] for side in pair.values() for v in side}.__len__()!=1:
            raise RuntimeError(f'Output mismatch: {target} {scenario} {count} {opcache}')
        def m(side,key):return statistics.median(v[key] for v in pair[side])
        print(f'{target} {scenario} {count}, OPcache={opcache}: peak used {m("before","peak_used"):.0f} -> {m("after","peak_used"):.0f}, delta {m("after","peak_used")-m("before","peak_used"):+.0f}; reserved {m("before","peak_reserved"):.0f} -> {m("after","peak_reserved"):.0f}; RSS {m("before","max_rss"):.0f} -> {m("after","max_rss"):.0f}',flush=True)
        pathlib.Path('/tmp/twig-memory-results.json').write_text(json.dumps(all_results,indent=2))