Repository navigation
Fix slow decoding of large MIME inputs on Linux; add opt-in exact sizing; benchmark MIME (CRLF) - #5
Merged
Merged
Conversation
Measuring in one process let earlier allocations raise glibc's mmap threshold, which hid the cost of allocating large outputs. Each row now runs in a child process that reads its input from a file, with fastpybase64 timed first; --same-process restores the old behavior.
MIME input decodes to fewer bytes than the maximal length, and shrinking the result with _PyBytes_Resize trapped glibc into mmap'ing fresh pages on every call. Decoding 1 MiB of CRLF MIME on a Xeon 6548N goes from 3.1 to 12.1 GB/s.
The default keeps the in-place shrink, which may leave up to 1/8 of the allocation unused. Exact sizing computes the length with binary_length_from_base64 and decodes with base64_to_binary_safe; it wastes no memory but decodes more slowly. The tests run in both modes, and bench.py gains --exact-sizing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On Linux (glibc), decoding large MIME input (76-character lines ending with CRLF) was about four times slower than it should be: 3.1 GB/s for 1 MiB on a Xeon 6548N, against 19.7 GB/s for the same data without line breaks.
try_decodesizes the output withmaximal_binary_length_from_base64, which counts line breaks as data, so MIME input always decodes to fewer bytes and the result is shrunk with_PyBytes_Resize. With glibc:mmap'ed, and the shrink is anmremap;strace shows one
mmap, onemremapand onemunmapper call, about 250 page faults per call for 1 MiB. Plain input never shrinks and is not affected; macOS malloc is not affected either.Fix
When the slack is at most 1/8 of the buffer, shorten the bytes object in place (
Py_SET_SIZEand a NUL terminator) instead of reallocating it. Every call then requests the same block size, the threshold settles on it, and the block is reused. The memory overhead is at most 1/8, about 2.6% for 76-character CRLF lines. Larger slack still goes through_PyBytes_Resize.Exact sizing (opt-in)
fastpybase64.set_exact_sizing(True)(withget_exact_sizing()) sizes decoded outputs exactly:simdutf::binary_length_from_base64computes the length, andbase64_to_binary_safedecodes into the buffer. No memory goes unused, but the extra pass makes decoding slower, so it is off by default. The setting is process-wide (astd::atomic<bool>, safe on free-threaded builds). The whole test suite now runs in both modes, andbench.pygains--exact-sizing. The README has a new Memory section.Xeon 6548N, GB/s of decoded data:
b64decode(plain), 1 MiBb64decode(plain), 16 MiBb64decode(MIME CRLF), 1 MiBb64decode(MIME CRLF), 16 MiBBenchmark
bench.pygains ab64decode(MIME CRLF)operation. Measuring everything in one process hid the problem: earlier allocations (building the input,random.randbytes, pybase64's temporary copy) raise glibc's threshold. Each row now runs in a fresh child process that reads its input from a file, with fastpybase64 timed first and the results checked afterwards.--same-processkeeps the old behavior.Results
Xeon Gold 6548N (Emerald Rapids), one core, Python 3.15.2,
icelakekernel, GB/s of decoded data:b64decode(MIME CRLF), 64 KiBb64decode(MIME CRLF), 1 MiBb64decode(MIME CRLF), 16 MiBdecodebytes, 1 MiBb64decode(plain), 1 MiBThe standard library suffers from the same trap: its
a2b_base64also over-allocates and shrinks.Tests (642, both modes) pass on Linux x64 and on macOS arm64 (Apple M4 Max). On the Mac, the numbers are unchanged within noise.