Repository navigation
Update the jar to Apache POI 5.5.1 and keep its Java source in java/ - #3
Merged
Merged
Conversation
poi-3.17 のアップグレードと RTF フォーマットの対応
Imported with git subtree from https://bitbucket.org/cmscom/msdoc2txt (commit 290d060, 2018-03-30), with its history. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> git-subtree-dir: java git-subtree-mainline: e77946b git-subtree-split: 290d060
java/check-output.sh runs msdoc2txt.jar on the samples in tests/samples and compares its stdout, byte for byte, with java/expected/<sample>.txt. It also requires exit code 0 and an empty stderr. The new jar must give the same output. The expected files were written with the current jar (Apache POI 3.17): document.doc 130 bytes, document.docx 125, empty.docx 0, presentation.pptx 56, presentation-notes.pptx 119, spreadsheet.xls 121 and spreadsheet.xlsx 119. .gitattributes keeps their line ends as they are (the .doc text has CRLF). presentation-notes.pptx is a new sample with speaker notes, so that the order of the output (notes before the slide text) is recorded too. It was written with the POI 3.17 classes of the jar (XMLSlideShow with one text box; on the notes slide only the body placeholder is kept, the template's date and slide number placeholders are removed), and <cp:revision> was removed from docProps/core.xml as for the other OOXML samples. It has no names in its metadata. test_java checks that the words of the slide and of the notes are searchable. Mutations caught by check-output.sh: one changed byte in an expected file (output differs), JAVA=false (exit code 1), a java that writes to stderr (stderr is not empty, 7 of 7). The new test fails when it is given presentation.pptx, which has no notes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
java/samples/ had two Microsoft sample documents whose metadata names a person; the tests use tests/samples/ instead. javasrc/README.md pointed to the Bitbucket repository, whose source is now in java/. The wheel and the sdist contain neither java/ nor javasrc/. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dependencies (all pinned): poi, poi-ooxml and poi-scratchpad 5.5.1, rtfparserkit 1.16.0 (now from Maven Central) and commons-cli 1.11.0. download-maven-plugin, git-commit-id-plugin and junit are gone. The bytecode targets Java 11 (--release 11). The jar is made with maven-shade-plugin 3.6.2: - ServicesResourceTransformer merges META-INF/services: poi, poi-ooxml and poi-scratchpad each register their extractors with ServiceLoader under the same file name. - META-INF/LICENSE and META-INF/LICENSE.txt of all jars are appended, because the licenses of POI (ECMA-376 schemas) and commons-math3 have terms beyond the Apache License; the repeated text is accepted. curvesapi (BSD) has no license file in its jar, so its license.txt from the 1.08 tag upstream is in java/licenses/ and goes into META-INF/LICENSE-curvesapi.txt. NOTICE files are merged. - module-info.class, META-INF/versions/** and signature files are left out. The manifest has Main-Class and Apache-POI-Version. - log4j2.component.properties in the root of the jar selects the simple provider of the Log4j API and turns it off; without it the API prints a warning to stdout, which would end up in the text. Java changes for POI 5: - .ppt: one HSLFSlideShowImpl is read for the metadata and wrapped in an HSLFSlideShow for the text. The extractor is not closed; the old code closed a PowerPointExtractor, which closed the file system before the text was read, so every .ppt failed. - One branch for slides: .ppt and .pptx both get a SlideShowExtractor (XSLFExtractor for .pptx); the notes come from the same extractor with slides off and notes on. - OOXML: POIXMLExtractorFactory().create(pkg), since ExtractorFactory has no createExtractor(OPCPackage) any more. New import locations. java/build.sh builds in maven:3.9.16-eclipse-temurin-21-noble, pinned by digest, with all plugins pinned and a fixed outputTimestamp. Three builds gave the same sha256: one with the same cache, one with an empty cache, umask 077 and a copy of java/ elsewhere, and one after touching the sources. The committed jar is replaced in the next commit. Checked with the built jar (21.9 MB, was 15.8 MB): check-output.sh is OK for all 7 recorded samples, byte for byte, with JDK 11.0.32 and 21.0.12 (pinned eclipse-temurin JRE images) and the local JDK 21; stderr is empty. presentation.ppt now gives text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The jar is built with java/build.sh from the source of the previous commit: 21881642 bytes (was 15,846,480), sha256 175a97e0be6113cd7f775c49bf9b37a8e04a119be17660f8c163b888751cb5e5. java/check-output.sh gives the same output, byte for byte, for the seven samples that were recorded with the old jar, and now also for the two .ppt samples, whose output is recorded here for the first time (107 and 123 bytes; the metadata lines come from the templates that made the files). stderr is empty for all nine. presentation-notes.ppt is a new sample: presentation-notes.pptx converted with LibreOffice in headless mode. POI cannot create notes in a .ppt. Its Current User stream and summary information have no names. The .ppt test no longer expects a failure: the xfail and the test of the old behaviour are gone, and both .ppt samples must give their slide text, and the notes for presentation-notes.ppt. RED: test_ppt_text_is_searchable failed for both .ppt samples with the old jar (msdoc2txt failed with exit code 1). Mutations caught by check-output.sh, each built from a changed copy of java/: closing the slide show before the text is read, as the old code did (both .ppt fail with exit code 1), and leaving out log4j2.component.properties (all nine fail: a Log4j line on stdout). Also checked once, not added to the repository: a .docx encrypted with POI's Encryptor and a random password gives exit code 1 and no output. check-output.sh now ends the stderr excerpt with a newline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new "jar" job builds msdoc2txt.jar with java/build.sh, that is in the Maven image pinned by digest, compares it with the committed jar (sha256 and cmp), and runs java/check-output.sh with Java 11 and Java 21 in eclipse-temurin JRE images, also pinned by digest (11.0.32_9-jre-noble and 21.0.12_8-jre-noble, both 2026-09-18). Checked locally with the same steps on a git archive of HEAD and an empty Maven cache: the same sha256 as the committed jar, and 9 of 9 samples OK with both Java versions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- java/README.md is rewritten: what msdoc2txt does and its output, the reproducible build with java/build.sh, java/check-output.sh, how to update a dependency, how the jar is put together (services, Log4j), and the licenses of the bundled libraries, with the source of the copied curvesapi license. - README: Apache POI 5.5.1 and Java 11 or later; .ppt files and speaker notes are tested and the known issue is gone; the Java source is in java/. The license section says that the jar contains libraries under the Apache License 2.0 (some with additional terms) and a BSD license, which are compatible with the package when it is used under the GPL version 3. - AGENTS.md: java/ in the layout, rebuilding and checking the jar after a change, and how the new samples were made. - news: +ppt.documentation (a known issue that was never released) is replaced by +ppt.bugfix; +poi5.feature and +java11.breaking. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- java/README.md: the jar keeps the license and notice files that the libraries ship; the SparseBitSet and RTF Parser Kit jars have none. pom.xml is part of the jar, so any change to it needs a rebuild. - java/pom.xml: outputTimestamp fixes the time stamps of the entries that the build makes; entries copied from the dependencies keep their own, and CI checks that the final jar is the same as the committed one. - AGENTS.md: wrap a long line. The comment is in pom.xml, which maven-jar-plugin copies into the jar, so the jar is rebuilt with java/build.sh: sha256 ef6ce5dd95767c4dbd5e47ae383ea607aecfb7dde9b522bab78cb5a88fa0dd73. Compared with the previous jar, entry by entry, only META-INF/maven/jp.cmscom.c2.transform.msoffice/msdoc2txt/pom.xml differs (same names, order, time stamps and attributes), and check-output.sh is OK for all 9 samples. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebuilds
msdoc2txt.jarwith Apache POI 5.5.1 and moves its Java source from Bitbucket (cmscom/msdoc2txt, history kept withgit subtree) intojava/..pptfiles can now be read.Changes
.ppt(the extractor closed the file system before the text was read)META-INF/services, all library license/notice files, and Log4j silenced (nothing on stderr, only text on stdout)java/build.sh);java/check-output.shcompares the output byte for byte withjava/expected/.ppttests enabled (no xfail); new samples with speaker notes (.pptx,.ppt)⚠ Behaviour changes
.pptfiles now give text instead of an empty result. Output for all other samples is byte-for-byte the same as with the old jarTests
All green: 13 tox environments + ruff + mypy, 53 passed, 0 skipped, 0 xfailed; 9 samples match on JDK 11 and 21; rebuilt jar sha256 matches.
Notes
🤖 Generated with Claude Code