binascii: Reduce module RAM overhead and compiled footprint. - #1157
Open
agatti wants to merge 1 commit into
Open
binascii: Reduce module RAM overhead and compiled footprint.#1157agatti wants to merge 1 commit into
agatti wants to merge 1 commit into
Conversation
This commit makes some changes to the `binascii` module to reduce its footpint once compiled, and marginally improve the time it takes to import the module itself. The decoding table is now precomputed as a 256 entries long `bytes` object rather than doing the table building at import time. The table was built as a list of integers, so there's some considerable overhead compared to a single `bytes` object. Decoding data would also do some unnecessary roundtrips between integers and characters for the table lookup to figure out the 6-bits sequence to use for a given character. The original code, despite having a function called "a2b_base64" does not work with strings, so the roundtrips were not needed to begin with. CPython's equivalent works with strings too, but the changes proposed here do not change the function's behaviour. There was some dead code in the encoder function, used to calculate the final size of the output buffer. The code uses a list anyway, so this is not needed. The tables are now built using `const()`, and made private (hence the minor version number being bumped up). This saves even more space when compiled. Finally, the base64 part of the test suite was updated a bit to cover more padding cases. These changes reduce the compiled module size by 204 bytes. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR makes some changes to the
binasciimodule to reduce its footpint once compiled, and marginally improve the time it takes to import the module itself.The decoding table is now precomputed as a 256 entries long
bytesobject rather than doing the table building at import time. The table was built as a list of integers, so there's some considerable overhead compared to a singlebytesobject.Decoding data would also do some unnecessary roundtrips between integers and characters for the table lookup to figure out the 6-bits sequence to use for a given character. The original code, despite having a function called "a2b_base64" does not work with strings, so the roundtrips were not needed to begin with. CPython's equivalent works with strings too, but the changes proposed here do not change the function's behaviour.
There was some dead code in the encoder function, used to calculate the final size of the output buffer. The code uses a list anyway, so this is not needed.
The tables are now built using
const(), and made private (hence the minor version number being bumped up). This saves even more space when compiled.Finally, the base64 part of the test suite was updated a bit to cover more padding cases.
These changes reduce the compiled module size by 204 bytes.
Testing
The existing CI tests suite still passes, and the test file also runs as expected with CPython once moved out of its original directory (so the MicroPython
binasciimodule isn't used instead).Trade-offs and Alternatives
binascii.a2b_base64could be modified to also work with strings, following CPython, at the expense of a slightly reduced savings figure.Generative AI
I did not use generative AI tools when creating this PR.