Conversation
13471ce to
ff6095a
Compare
There was a problem hiding this comment.
Pull request overview
This PR fixes a potential buffer overflow when Prism.parse_stream reads from a Ruby IO via gets(limit), where gets may exceed the byte limit to avoid splitting a multi-byte character.
Changes:
- Reduce the
getslimit used byparse_stream_fgetsto leave headroom for multi-byte characters. - Add a regression test that exercises a long line with a multi-byte character at the boundary.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| test/prism/api/parse_stream_test.rb | Adds a test covering long-line streaming with a multi-byte boundary case. |
| ext/prism/extension.c | Adjusts the gets limit to avoid overfilling the fixed-size internal line buffer. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # case. gets(4095) may return 4095 or more bytes if the last | ||
| # character is a multi-byte character. For example, | ||
| # StringIO.new("\u3042").gets(1).bytesize is 3 not 1. | ||
| # parse_stream_fgets() uses |
Comment on lines
+1084
to
+1088
| #define MAX_ENC_LEN 6 | ||
| RUBY_ASSERT(size > MAX_ENC_LEN); | ||
|
|
||
| VALUE line = rb_funcall((VALUE) stream, rb_intern("gets"), 1, INT2FIX(size - 1)); | ||
| VALUE line = rb_funcall((VALUE) stream, rb_intern("gets"), 1, INT2FIX(size - MAX_ENC_LEN - 1)); | ||
| #undef MAX_ENC_LEN |
…al buffer `pm_source_stream_read()` has 4096 bytes internal buffer to read a line: https://github.com/ruby/prism/blob/39fb41f06e5f4ccbe1783122fa8a4470e09dfcfe/src/source.c#L368-L369 It uses `gets(4095)` to read the next chunk to the buffer: https://github.com/ruby/prism/blob/39fb41f06e5f4ccbe1783122fa8a4470e09dfcfe/ext/prism/extension.c#L1076 But `gets(4095)` may read 4095 or more data when the character at 4095 is a multi-byte character. In the case, `parse_stream_fgets()` writes 4097 or more data to the buffer. We can avoid it by using `gets(4095 - 6)` not `gets(4095)`. `6` is the max character length in all available encodings. We can reduce it by using `stream.external_encoding` and `rb_enc_mbmaxlen()` but it may slow down. So I used constant `6` in this change.
Earlopain
reviewed
Jul 7, 2026
| * max character length in all available encodings by the | ||
| * following command line in the Ruby source directory: | ||
| * | ||
| * grep -Eh "max (enc|byte) length" enc/*.c | |
Collaborator
There was a problem hiding this comment.
../../../../ext/prism/extension.c:1079:46: warning: ‘/*’ within comment [-Wcomment]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
pm_source_stream_read()has 4096 bytes internal buffer to read a line:prism/src/source.c
Lines 368 to 369 in 39fb41f
It uses
gets(4095)to read the next chunk to the buffer:prism/ext/prism/extension.c
Line 1076 in 39fb41f
But
gets(4095)may read 4095 or more data when the character at 4095 is a multi-byte character. In the case,parse_stream_fgets()writes 4097 or more data to the buffer.We can avoid it by using
gets(4095 - 6)notgets(4095).6is the max character length in all available encodings. We can reduce it by usingstream.external_encodingandrb_enc_mbmaxlen()but it may slow down. So I used constant6in this change.