Repository navigation
fix(compress): stop pass B from destroying content - #207
Merged
Merged
Conversation
PyMuPDF 1.28.2 made Document.scrub() remove invisible text again, and
pass B of Compress called it with its defaults. The v1.15.0 and v1.15.1
binaries ship 1.28.2, so they deleted the OCR layer when compressing.
Measured by QA on PDFApps' own OCR output: in 17 of 18 level and
Ghostscript combinations the old code left 0 of 140 words; with this
change all 18 of 18 keep the 140 words, still invisible.
With its defaults scrub() also removed links, form values (reset to
their defaults), attachments, JavaScript, replies and popups, metadata
and XMP, thumbnails, and visible text overlapping the invisible text.
It also applied pending redactions, and it could raise halfway through,
leaving a half-cleaned document that was then saved.
Pass B no longer calls scrub() and saves with clean=False. clean=True
duplicated glyphs on the pymupdf#5054 construction ("(f)Tj 0 0 TD
(faeces)Tj" became "ffaeces") on PyMuPDF 1.28.0 and 1.28.2.
The value of progress.compress.passB_setup changed in all 8 languages.
Tests: 27 new cases in tests/test_compress_preserves_content.py; 25 of
them fail against the old code.
Known limitation: with Ghostscript installed, pass A can still produce
the winning output, and that output can lose form fields, attachments,
links, OCGs, the structure tree and pending redactions. This is left
for later increments.
Size: output can be larger because metadata is now preserved. Measured
range: -5.5% to +53%. The +53% was a PDF with a 14 KB JPEG thumbnail
embedded in its XMP. On PDFs with OCR, +0.3% to +0.5%.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
…s bound CodeQL (py/uninitialized-local-variable) flagged `langs` in _tesseract_or_skip as possibly used before assignment: it is bound inside the try, and the except only calls pytest.skip, which CodeQL does not know always raises. The alert is a false positive. Moving the "eng" check into an else on the try makes the flow explicit: it runs only when get_languages returned. Behaviour is unchanged: with Tesseract installed the test still runs and passes, and the three skip paths (no executable, get_languages failing, no 'eng' data) still skip with the same messages. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Deploying pdfapps with
|
| Latest commit: |
ae04edd
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://c7816f65.pdfapps.pages.dev |
| Branch Preview URL: | https://fix-compress-preserve-pass-b.pdfapps.pages.dev |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PyMuPDF 1.28.2 made
Document.scrub()remove invisible text again, and pass B of Compress called it with its defaults. The v1.15.0 and v1.15.1 binaries ship 1.28.2, so they deleted the OCR layer when compressing.With its defaults
scrub()also removed links, form values (reset to their defaults), attachments, JavaScript, replies and popups, metadata and XMP, thumbnails, and visible text overlapping the invisible text. It also applied pending redactions, and it could raise halfway through, leaving a half-cleaned document that was then saved.Pass B no longer calls
scrub()and saves withclean=False.clean=Trueduplicated glyphs on the pymupdf#5054 construction ((f)Tj 0 0 TD (faeces)Tjbecame "ffaeces") on PyMuPDF 1.28.0 and 1.28.2.The value of
progress.compress.passB_setupchanged in all 8 languages.Files:
app/utils.py,app/translations.json,tests/test_compress_preserves_content.py(new, 27 cases).Measurements
Known limitation
With Ghostscript installed, pass A can still produce the winning output, and that output can lose form fields, attachments, links, OCGs, the structure tree and pending redactions. This is left for later increments.
Size
Output can be larger because metadata is now preserved. Measured range: -5.5% to +53%. The +53% was a PDF with a 14 KB JPEG thumbnail embedded in its XMP. On PDFs with OCR, +0.3% to +0.5%.
Verification
scrub()with its defaults fails 25;clean=Truealone fails 2.scrub()flag on its own makes the test named after that content fail.test_real_tesseract_ocr_layer_survivesskips because Tesseract is not installed on the runner.🤖 Generated with Claude Code