Skip to content

ext/standard: Decode base64 four characters at a time in the character loop - #24774

Open
ArtUkrainskiy wants to merge 1 commit into
php:masterfrom
ArtUkrainskiy:base64-decode-quads
Open

ArtUkrainskiy wants to merge 1 commit into
php:masterfrom
ArtUkrainskiy:base64-decode-quads

Conversation

@ArtUkrainskiy

@ArtUkrainskiy ArtUkrainskiy commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

base64_decode() runs a character loop in php_base64_decode_impl(): one character per iteration, with a switch on its position in the group. That loop is the whole decoder for input under 45 bytes — tokens, ids, keys — and for everything after the first space or line break, where the vector decoders (SSSE3/AVX2/AVX-512/NEON) stop.

This PR decodes whole groups of four alphabet characters at once while the input is clean, and keeps the one-character step for whitespace, padding, invalid bytes and the last partial group. The vector decoders are untouched. Output is unchanged, including every strict-mode rejection.

Release builds, master vs this PR. Instructions per call (perf stat, deterministic); the range spans six compilers — GCC 13/14 and clang 18/20 on Intel, GCC 15 and clang 21 on AMD:

base64_decode() input instructions
24 bytes (a token) 41–46% fewer
44 bytes (a 32-byte key) 45–50% fewer
1 KB, clean 29–31% fewer
64 KB, clean unchanged
64 KB in 76-column lines (MIME) 69–71% fewer
64 KB in 64-column lines (PEM) 68–71% fewer
64 KB MIME, strict 73–76% fewer

In CPU cycles on the AMD host the MIME and PEM cases are 63–81% fewer, short input 34–64% fewer. base64_encode() is not touched. Clean 64 KB barely moves: the vector loop does the bulk, and only the ≤44-byte tail reaches this loop.

The short rows are any base64 under ~45 bytes, decoded entirely by this loop. The line-wrapped rows apply when a whole wrapped blob goes to base64_decode() in one call: it skips the line breaks, even in strict mode, and runs the rest through this loop. Email base64 is wrapped at 76 columns (RFC 2045), PEM at 64. A parser that strips the breaks first stays on the vector path and is unaffected.

Why it is safe: a group is decoded only when all four characters are in the alphabet, so whitespace, padding and invalid bytes still take the one-character path and every strict-mode rejection is preserved. A differential test against master decoded 30,000 random inputs on each of the six compilers — clean, MIME and PEM lines, whitespace and invalid bytes anywhere, padding in every position, both modes — with identical output; the same test flags a deliberately broken build, so it is not toothless. The change also matches master under a forced-scalar build and under ASan + UBSan (80,000 decodes). A new .phpt covers a line break at every position of short input in both modes.

Not tested: 32-bit and ZTS by reading only; the AVX-512 and NEON paths are unchanged and not re-run here.

Full numbers and reproduction: https://github.com/ArtUkrainskiy/php-src-bench/tree/main/reports/base64-decode-quads

A follow-up could resume the vector loop after a line break instead of handing the rest to this loop — about 3× more on MIME and PEM — but that touches every vector variant, so I will send it separately.

…r loop

The character loop of base64_decode() handled one character per
iteration with a switch on its position in the group, and it is the
whole decoder for input shorter than 45 bytes and for everything after
the first whitespace or line break, where the vector loops stop.

Decode whole groups of four alphabet characters at once while the input
is clean, and keep the one-character step for whitespace, padding,
invalid bytes and the last partial group. Output is unchanged.

Short input runs with about 40-50% fewer instructions, base64 with MIME
line breaks about 70% fewer.
@ArtUkrainskiy
ArtUkrainskiy marked this pull request as ready for review October 27, 2026 05:08
@ArtUkrainskiy
ArtUkrainskiy requested a review from bukka as a code owner October 27, 2026 05:08
@Sjord

Sjord commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

Looks good to me.

If you want to take this further, @lemire and @WojciechMula did impressive research in making base64 decoding very fast.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants