Fix Encoding::InvalidByteSequenceError in Aliki on non-UTF-8 locales

[ybiquitous]

RDoc::Generator::Darkfish reads template files with Encoding.default_external. As a result, Aliki's templates, which contain non-ASCII characters such as emojis, raised Encoding::InvalidByteSequenceError when the external encoding was UTF-8, such as with LANG=C.

This adds a #template_encoding hook to Generator::Darkfish with a default value of nil, and lets Generator::Aliki override the hook with Encoding::UTF_8.

As a result, Aliki template files are always read with UTF-8.

Fixes #1574

[Copilot]

Pull request overview

This PR addresses Encoding::InvalidByteSequenceError when generating Aliki output under non-UTF-8 default external encodings (e.g., LANG=C) by introducing a generator hook to control the encoding used when reading template files.

Changes:

• Add a #template_encoding hook to RDoc::Generator::Darkfish, and use it when reading template files.
• Override #template_encoding in RDoc::Generator::Aliki to force UTF-8 template reads.
• Add a regression test ensuring Aliki generation succeeds and preserves non-ASCII characters when Encoding.default_external is non-UTF-8.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
test/rdoc/generator/aliki_test.rb	Adds a regression test covering generation under a non-UTF-8 Encoding.default_external.
lib/rdoc/generator/darkfish.rb	Introduces template_encoding hook and uses it for template file reads.
lib/rdoc/generator/aliki.rb	Overrides template_encoding to force UTF-8 when reading Aliki templates.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[ybiquitous]

Indeed, this is not a problem. nil means to leave Ruby the default encoding, rather than specifying Encoding.default_external.

nil also works well. Here's some evidence.

Save test_encoding.rb with:

filename = ARGV[0] or abort "specify file"

puts "# Ruby #{RUBY_VERSION}"

f1 = File.read(filename, encoding: nil)
f2 = File.read(filename)
f3 = File.read(filename, encoding: Encoding::UTF_8)

printf "nil == no-arg: %s\n", f1.encoding == f2.encoding && f1 == f2
printf "valid_encoding?: nil-read=%s, utf8-read=%s\n", f1.valid_encoding?, f3.valid_encoding?

begin
  f1.encode(Encoding::UTF_8)
rescue EncodingError => e
  puts "f1.encode(UTF-8) -> #{e.class}"
end

Run (the same output across the Ruby versions):

$ echo '😄' > sample.txt

$ ruby -E US-ASCII test_encoding.rb sample.txt
# Ruby 4.0.6
nil == no-arg: true
valid_encoding?: nil-read=false, utf8-read=true
f1.encode(UTF-8) -> Encoding::InvalidByteSequenceError

$ rbenv shell 3.4.7
$ ruby -E US-ASCII test_encoding.rb sample.txt
# Ruby 3.4.7
nil == no-arg: true
valid_encoding?: nil-read=false, utf8-read=true
f1.encode(UTF-8) -> Encoding::InvalidByteSequenceError

$ rbenv shell 3.3.12
$ ruby -E US-ASCII test_encoding.rb sample.txt
# Ruby 3.3.12
nil == no-arg: true
valid_encoding?: nil-read=false, utf8-read=true
f1.encode(UTF-8) -> Encoding::InvalidByteSequenceError

$ rbenv shell 3.2.11
$ ruby -E US-ASCII test_encoding.rb sample.txt
# Ruby 3.2.11
nil == no-arg: true
valid_encoding?: nil-read=false, utf8-read=true
f1.encode(UTF-8) -> Encoding::InvalidByteSequenceError

[kou]

+1