Skip to content

Fix Emacs-Mule codeToMbcLength, used by stateless-ISO-2022-JP too - #71

Merged
headius merged 1 commit into
jruby:masterfrom
sampokuokkanen:emacs-mule-code-length
Nov 11, 2026
Merged

headius merged 1 commit into
jruby:masterfrom
sampokuokkanen:emacs-mule-code-length

Conversation

@sampokuokkanen

Copy link
Copy Markdown
Contributor

The code is an unsigned 32-bit value held in a Java int. 0xffffffff is -1 as an int, so code > 0xffffffff was true for every non-negative code, and every non-ASCII code got length 0. Test the high bit of each byte instead, as CRuby's code_to_mbclen in enc/emacs_mule.c does with unsigned compares.

This also fixes the stateless-ISO-2022-JP encodings, which share this implementation. JRuby looped forever in String#tr and #codepoints on them, and Integer#chr rejected valid codes.

e = "stateless-ISO-2022-JP"

"あいう".encode(e).tr("あ".encode(e), "う".encode(e)).encode("UTF-8")
# MRI:         "ういう"
# JRuby master: never returns (eventually OutOfMemoryError)

"あ".encode(e).ord.chr(e).encode("UTF-8")
# MRI:         "あ"
# JRuby master: ArgumentError: invalid codepoint 0x92a4a2 in stateless-ISO-2022-JP

The code is an unsigned 32-bit value held in a Java int. 0xffffffff is
-1 as an int, so `code > 0xffffffff` was true for every non-negative
code, and every non-ASCII code got length 0. Test the high bit of each
byte instead, as CRuby's code_to_mbclen in enc/emacs_mule.c does with
unsigned compares.

This also fixes the stateless-ISO-2022-JP encodings, which share this
implementation. JRuby looped forever in String#tr and #codepoints on
them, and Integer#chr rejected valid codes.

@headius headius left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, silly mistake somewhere along the way.

@headius headius added this to the 1.0.65 milestone Nov 11, 2026
@headius
headius merged commit 38bab2b into jruby:master Nov 11, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants