fix: close five more open upstream issues and adopt two stalled PRs - #2
Merged
Conversation
A sweep of everything still open on iarna/rtf-parser that is a defect rather than a feature request. Each change carries a regression test using the reporter's own reproduction where they supplied one. interpreter: accept both the signed and unsigned forms of \uN The spec writes a code unit above 32767 as a negative number; plenty of producers emit the unsigned value instead, and writeInt16LE threw ERR_OUT_OF_RANGE on those -- an uncaught crash rather than a wrong character. A negative value is now normalised to its unsigned counterpart and anything outside the UTF-16 range is skipped rather than thrown. Adopts upstream PR iarna#28 by @UKayeF, whose diagnosis was right; the naive swap to writeUInt16LE in that PR would have broken the SIGNED form instead (writeUInt16LE(-1) throws), so this handles both. Closes: iarna#28. Also fixes the emoji break reported in iarna#15: \u55357 \u56842 is a surrogate pair, and both halves exceed 32767. interpreter: \fcharset238 is CP1250, and \ansicpg1256 is available 'CP238' was never a code page and iconv has never known it, so a document with an east-European font threw on its first hex escape. 1256 was mapped by font (\fcharset178) but rejected as a document code page. Adopts upstream PR iarna#33 by @facue. Closes: iarna#30, iarna#33. interpreter: decode the Symbol font instead of throwing on it \fcharset2 is a glyph set, not a code page -- iconv has no 'SYMBOL' encoding, so a Symbol font (which is how Word writes bullets) took the whole document down. Adds an Adobe Symbol table for the 0xA0..0xFE range where the list glyphs live, so \'b7 becomes a real bullet. Any charset iconv does not recognise now falls back to CP1252 with a debug note rather than aborting: one exotic font table should not cost the reader the entire file. Closes: iarna#15 (the reported break and the bullet case; the 0x20..0x7E Greek half is deliberately left unmapped and documented as such). interpreter: skip \pict payloads instead of emitting them as text RTF stores an image as a hex payload inside {\pict ...}. With no handler it was decoded as ordinary text, so a document's content gained a span of raw hex -- 'before 89504e470d0a...' -- corrupting text extraction and word counts. Marked ignorable, exactly as \stylesheet and \info already are. Closes: iarna#32. deps: drop readable-stream, iconv-lite@^0.7.3 Only Writable and Transform were used, both built into node for a decade, so this REMOVES the dependency rather than bumping it. Runtime dependencies go from two to one, and the remaining one is current (0.4.24 dated from 2018). Closes: iarna#37. Not addressed here, deliberately: iarna#35 tables -- a real feature, not a defect, and a large one. iarna#34 -- a support question about callbacks, not a bug in the parser. iarna#29 -- superseded: this fork already emits those six control words plus eight more. @wwwildcat got there first and deserves the credit. Verified: 19 tests / 44 assertions green across both suites, lint clean, and the 315k-character manuscript still extracts to exactly its \nofchars count (314,892) with zero replacement characters. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sweeps every remaining open defect on iarna/rtf-parser and adopts two stalled upstream pull requests. Where a reporter supplied a reproduction, that exact input is the test.
What this closes
\fcharset2) threw — and it's how Word writes bullets\'b7→•, list text survivesERR_OUT_OF_RANGE\u55357 \u56842decodes to one code point\u61607threw\fcharset238→'CP238', which threw\'e8→č"before 89504e470d0a… after""before after"\ansicpg1256rejected as unavailable\'c7→اiconv-lite@0.4.15(2018),readable-stream@2iconv-lite@^0.7.3,readable-streamremovedNote how many of these were crashes, not wrong output — a single exotic font table or one out-of-range escape aborted the entire document. Unknown charsets now fall back to CP1252 with a debug note instead of throwing.
Credit where it's owed
writeUInt16LEwould have broken the signed form the spec actually prescribes (writeUInt16LE(-1)throws), so this normalises negatives first and handles both.Deliberately not done
\trowd/\cell/\row/\intbl) need a new node type and real design.Dependency change
readable-streamused onlyWritableandTransform, both built into node for a decade — so this removes the dependency rather than bumping it. Runtime deps: 2 → 1. Verified withnpm ls readable-stream --omit=devreturning empty.Verification
test/encoding.js,test/upstream-issues.js),standardclean,npm testexit 0{\nofchars314892}header count, 1,966 apostrophes / 1,943 quotes / 308 en dashes, zero replacement charactersScope note
The Symbol table covers 0xA0–0xFE, where the list glyphs live. The 0x20–0x7E half maps ASCII positions onto Greek and is left unmapped — documented in the source. It never throws; unmapped bytes become U+FFFD.