Repository navigation
Port to Python 3 and Plone 6 - #1
Merged
Merged
Conversation
Move the code to src/c2/transform/msoffice/ with git mv so that history follows the files, and remove c2/__init__.py and c2/transform/__init__.py to make c2 and c2.transform PEP 420 namespace packages. Also move README.txt to README.md, docs/HISTORY.txt to CHANGELOG.md and the license files to the top level, and remove .hgignore. File contents are unchanged; they are rewritten in later commits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Packaging: - Replace setup.py, setup.cfg and the old MANIFEST.in with pyproject.toml (setuptools, src/ layout, PEP 420 namespaces) and a MANIFEST.in that grafts src/c2, so the jar is shipped. - Version 2.0.0.dev0, Plone 6.0-6.2, Python 3.10-3.14. - tox: ruff-check, py310-313 with Plone 6.0 and 6.1, py310-314 with Plone 6.2, installed with pip against the Plone constraints. pytest runs with -rs. C2_TRANSFORM_MSOFFICE_REQUIRE_JAVA and JAVA_HOME are passed on. - towncrier (news/) and zest.releaser settings. Transform: - Run java with subprocess.run: java, -Xmx512m, -Dfile.encoding=UTF-8, -Dstdout.encoding=UTF-8, -jar, the jar. Timeout 60 seconds. - C2_TRANSFORM_MSOFFICE_TIMEOUT and C2_TRANSFORM_MSOFFICE_JAVA_OPTS change the timeout and the JVM options; the encoding options always come last. - The exit code decides success. Exit code 0 with empty output is a document without text. Output is decoded with errors="replace". - On failure the text is empty and the error is logged with at most 500 characters of stderr, never the document. - str input is logged and gives empty text instead of an assert. - Add application/vnd.ms-powerpoint to inputs. Drop the Windows and Plone 3 code paths. config.py keeps adding the Office types to the standard mimetypes module on import, now also imported from the package's __init__. setuphandlers.py only gets an absolute import and formatting here so that ruff passes; it is rewritten with tests in the next commit. The old PloneTestCase test is removed. RED: - tests/test_imports.py: ModuleNotFoundError: No module named 'config'. - tests/test_transform.py: ImportError: cannot import name 'implements' from 'zope.interface'. - tests/test_config.py on py312-Plone62 with add_type commented out: docm, pptm, pptx, xlsb, xlsm came back as None. Mutations of transform.py, each caught by tests/test_transform.py: dropping the encoding options, putting them before the JVM options, deciding by stderr, strict decoding, dropping the str guard, logging the whole stderr, accepting a timeout of 0, logging empty output as an error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Register the default profile with a GenericSetup post_handler instead of the old import step. It registers the Office MIME types that the registry does not know yet, and adds the transform unless portal_transforms already has an object with its name (then it logs and skips, as before). Running it again changes nothing. - The profile version is 2000. There is no upgrade step from 1.0b6; 2.0 is meant to be installed again. - Add an uninstall profile whose post_handler removes the transform if it is there. If unregisterTransform raises KeyError (a MIME type that was never mapped), log a warning and go on. MIME types are kept. - Hide the uninstall profile with INonInstallable. - The test layer now applies the default profile. RED: tests/test_setup.py failed 7 of 11 with the old handler: profile version '1' != '2000', no INonInstallable utility, "no uninstall profile found", and the second install was not logged. Mutations of setuphandlers.py, each caught: no objectIds guard on uninstall, catching the wrong exception, no objectIds guard on install, never adding a missing MIME type. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace wordprocessing1.docx (taken from another project) with samples made for this package: document.docx/.doc with a short English sentence containing the words the old test looked for and a Japanese sentence, spreadsheet.xlsx/.xls, presentation.pptx/.ppt and an empty.docx. tests/test_java.py runs the bundled jar with LC_ALL=C, as in a slim container image: - docx, doc, xls, xlsx and pptx text, including Japanese, ends up in the File's SearchableText. - A document without text gives empty text and no log. - Without java on PATH, creating the File still works and an error is logged. - The tests are skipped without java, and fail instead when C2_TRANSFORM_MSOFFICE_REQUIRE_JAVA=1. Known issue: the bundled jar (POI 3.17) cannot read any .ppt file, as in 1.0b6. One test pins the current behaviour (empty text, an error log, the File is saved) and a strict xfail test expects the text, so it starts failing once the jar is fixed. Checks: - Without java: 10 skipped; with C2_TRANSFORM_MSOFFICE_REQUIRE_JAVA=1: 9 errors. - Removing the -Dfile.encoding/-Dstdout.encoding options from the java command makes 6 tests fail (Japanese text becomes "?"). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- lint: tox -e ruff-check on Python 3.12. - test: the tox matrix (Plone 6.0 and 6.1 on Python 3.10-3.13, Plone 6.2 on Python 3.10-3.14) with Temurin 21 from actions/setup-java. - C2_TRANSFORM_MSOFFICE_REQUIRE_JAVA=1 turns a missing java into test failures, and a last step fails the job if the pytest output (-rs) reports any skipped test. - Actions are pinned to commit SHAs of releases at least two weeks old; permissions are contents: read. Checked with actionlint 1.7.12. The skip check was tried on a log with skipped tests (matches) and on a clean log (no match). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- README.md: what the package does, supported Plone, Python and Java versions, installation, upgrading from 1.0b6 (reinstall, PEP 420 and horse-with-no-namespace), how the jar is called, the environment variables, memory use (one JVM per Zope thread), the .ppt known issue, uninstall, credits and contributors. It absorbs docs/INSTALL.txt. - CHANGELOG.md: the old docs/HISTORY.txt in Markdown, below the towncrier start marker. The 2.0.0 changes are news fragments. - AGENTS.md: rules (English, no specs in the repository, news fragments), layout, how to run tox, and how the test samples were made. Checked: towncrier build --draft renders the fragments; uv build makes a wheel with msdoc2txt.jar and without tests/ or .github/; twine check passes for the wheel and the sdist. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_uninstall_profile_is_hidden used no fixture, so the ZCML of the package was loaded only if another test had set up the layer before it. On Plone 6.1 it ran without the layer and failed with ComponentLookupError. It now requests the portal fixture. RED: py310-py313 with Plone61 failed with ComponentLookupError for INonInstallable 'c2.transform.msoffice'. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
shlex.split raises ValueError for options such as an unclosed quote. It was logged as "msdoc2txt could not run" with a traceback; it is now logged as "JVM options in C2_TRANSFORM_MSOFFICE_JAVA_OPTS could not be parsed" and java is not started. The text is empty, as for other failures. RED: test_java_opts_that_cannot_be_parsed got ['msdoc2txt could not run']. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tests/test_java.py imports plone.api. It was only installed because another test dependency pulls it in. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"Supported files" listed every extension, although only .docx, .doc, .xlsx, .xls and .pptx are covered by the tests and .ppt gives no text. The README now lists the MIME types the transform is registered for, and the tested formats separately. AGENTS.md now says that the last editor name in the .ppt sample's Current User stream was cleared, and to check new samples for names. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A failed run of the jar logged the first 500 characters of its stderr. A Java exception can quote parts of the document, such as the name of a part, so the log now has only the exit code. STDERR_LOG_LIMIT is gone. test_stderr_is_not_logged puts a part name into stderr and checks that it is not in the log, also at DEBUG level. RED: test_non_zero_exit_code, test_stderr_is_not_logged and test_ppt_gives_no_text_with_the_bundled_jar failed with the stack trace in the message. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TransformEngine.unregisterTransform unmaps the transform before it deletes it. When a MIME type mapping was missing, _unmapTransform raised KeyError, uninstall only logged a warning and the transform stayed, so installing again skipped it as already registered. On KeyError the transform is now deleted with _delObject and suppress_events=True: Transform.manage_beforeDelete would unmap again, and OFS logs that KeyError as an error and re-raises it in debug mode for users without the Manager role. The tests delete the DOCX -> text/plain mapping of a real site instead of replacing unregisterTransform: uninstall removes the transform and logs one warning, and installing again maps it. RED: test_missing_mapping_still_removes_the_transform (transform still in objectIds) and test_install_after_a_missing_mapping_adds_the_transform (KeyError: install was skipped). Mutations caught: no deletion (2 failed), suppress_events=False (1 failed: "_delObject() threw" logged). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
C2_TRANSFORM_MSOFFICE_TIMEOUT was checked with str.isdigit(), which
accepts "²", and int("²") then raised ValueError out of the transform.
The value is now parsed with int() directly; a ValueError or a value
that is not positive logs the warning and uses the default.
RED: test_invalid_timeout_falls_back_to_default["²"] failed with
"ValueError: invalid literal for int() with base 10: '²'". Mutation
caught: "seconds >= 0" (4 failed).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports c2.transform.msoffice from Python 2 / Plone 4–5 to Python 3.10–3.14 / Plone 6.0–6.2 and prepares the 2.0.0 release. The history from Bitbucket is kept;
1.0b6is tagged onf057e89.Changes
src/with a PEP 420 namespace; replacesetup.pywithpyproject.toml; add tox (Plone 6.0–6.2 × supported Pythons), pytest-plone, towncrier and zest.releasersubprocess.runwith a timeout,-Dfile.encoding=UTF-8 -Dstdout.encoding=UTF-8, success decided by the exit code (only the exit code is logged on failure),C2_TRANSFORM_MSOFFICE_TIMEOUT/C2_TRANSFORM_MSOFFICE_JAVA_OPTSpost_handler; add an uninstall profile (hidden)docs/HISTORY.txt) and AGENTS.md⚠ Behaviour changes
2000; there is no upgrade step from 1.0b6 (reinstall the add-on)Tests
All green: 13 tox environments + ruff, 51 passed / 1 xfailed, 0 skipped (with Java).
Notes
.ppt; this was already the case in 1.0b6. The test isxfail(strict=True)until the jar is updated.🤖 Generated with Claude Code