Repository navigation
FIX: decode variation selector payloads next to emoji with presentation selectors - #3141
Open
Jaafer Rahmani (bIackr0se) wants to merge 1 commit into
Open
Jaafer Rahmani (bIackr0se) wants to merge 1 commit into
Jaafer Rahmani (bIackr0se) wants to merge 1 commit into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
VariationSelectorSmugglerConverter(action="decode")returns'\x0f'instead of the hidden text when an ordinary emoji such as ❤️ or ✔️ comes before the payload. These emoji are written as the symbol plus U+FE0F, a variation selector that picks the emoji presentation. The decoder took the first variation selector anywhere in the text as the start of the payload, decoded that U+FE0F as byte0x0Fand stopped at the next ordinary character.decode_visible_hiddenhad a related problem: it split at the first base character (😊 by default), so visible text that contained 😊 was cut short.The payload
secret, encoded with the default settings and placed after different carrier text:The results are the same with
embed_in_base=False.The fix:
decode_messagestarts at the variation selectors that follow the converter's base character, with or without the space separator. Text that has no such payload is scanned as before, so a payload encoded with a different base character still decodes.decode_visible_hiddensplits at the last base character, which is the oneencode_visible_hiddenappends after the visible text (with a space as the base character andembed_in_base=False, the space before the separator). The payload holds only variation selectors, so it cannot contain the base character.Tests and Documentation
test_token_smuggler_converter.py: decoding with ❤️, ✔️ and 😊 before the payload, in both embedding modes; visible/hidden round trips with 😊 and ❤️ in the visible text and with an empty hidden text; decoding with the otherembed_in_basesetting; a space as the base character with the separator; decoding a payload encoded with another base character. 20 of the new cases fail onmain.tests/unit/converterpasses (2094 passed, 34 skipped). ruff is clean, and ty (0.0.84) reports the same diagnostics as onmain.