Repository navigation
ext/standard: Decode base64 four characters at a time in the character loop - #24774
Open
ArtUkrainskiy wants to merge 1 commit into
Open
ArtUkrainskiy wants to merge 1 commit into
ArtUkrainskiy wants to merge 1 commit into
Conversation
…r loop The character loop of base64_decode() handled one character per iteration with a switch on its position in the group, and it is the whole decoder for input shorter than 45 bytes and for everything after the first whitespace or line break, where the vector loops stop. Decode whole groups of four alphabet characters at once while the input is clean, and keep the one-character step for whitespace, padding, invalid bytes and the last partial group. Output is unchanged. Short input runs with about 40-50% fewer instructions, base64 with MIME line breaks about 70% fewer.
ArtUkrainskiy
force-pushed
the
base64-decode-quads
branch
from
October 27, 2026 04:50
835f8e0 to
527f18f
Compare
ArtUkrainskiy
marked this pull request as ready for review
October 27, 2026 05:08
Contributor
|
Looks good to me. If you want to take this further, @lemire and @WojciechMula did impressive research in making base64 decoding very fast. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
base64_decode() runs a character loop in
php_base64_decode_impl(): one character per iteration, with aswitchon its position in the group. That loop is the whole decoder for input under 45 bytes — tokens, ids, keys — and for everything after the first space or line break, where the vector decoders (SSSE3/AVX2/AVX-512/NEON) stop.This PR decodes whole groups of four alphabet characters at once while the input is clean, and keeps the one-character step for whitespace, padding, invalid bytes and the last partial group. The vector decoders are untouched. Output is unchanged, including every strict-mode rejection.
Release builds, master vs this PR. Instructions per call (
perf stat, deterministic); the range spans six compilers — GCC 13/14 and clang 18/20 on Intel, GCC 15 and clang 21 on AMD:base64_decode()inputIn CPU cycles on the AMD host the MIME and PEM cases are 63–81% fewer, short input 34–64% fewer.
base64_encode()is not touched. Clean 64 KB barely moves: the vector loop does the bulk, and only the ≤44-byte tail reaches this loop.The short rows are any base64 under ~45 bytes, decoded entirely by this loop. The line-wrapped rows apply when a whole wrapped blob goes to
base64_decode()in one call: it skips the line breaks, even in strict mode, and runs the rest through this loop. Email base64 is wrapped at 76 columns (RFC 2045), PEM at 64. A parser that strips the breaks first stays on the vector path and is unaffected.Why it is safe: a group is decoded only when all four characters are in the alphabet, so whitespace, padding and invalid bytes still take the one-character path and every strict-mode rejection is preserved. A differential test against master decoded 30,000 random inputs on each of the six compilers — clean, MIME and PEM lines, whitespace and invalid bytes anywhere, padding in every position, both modes — with identical output; the same test flags a deliberately broken build, so it is not toothless. The change also matches master under a forced-scalar build and under ASan + UBSan (80,000 decodes). A new
.phptcovers a line break at every position of short input in both modes.Not tested: 32-bit and ZTS by reading only; the AVX-512 and NEON paths are unchanged and not re-run here.
Full numbers and reproduction: https://github.com/ArtUkrainskiy/php-src-bench/tree/main/reports/base64-decode-quads
A follow-up could resume the vector loop after a line break instead of handing the rest to this loop — about 3× more on MIME and PEM — but that touches every vector variant, so I will send it separately.