Skip to content

#37 : Add some tests for Asciidoc and Quickbook format.#38

Merged
wpak-ai merged 17 commits intocppalliance:developfrom
AuraMindNest:fix/add-tests
May 8, 2026
Merged

#37 : Add some tests for Asciidoc and Quickbook format.#38
wpak-ai merged 17 commits intocppalliance:developfrom
AuraMindNest:fix/add-tests

Conversation

@AuraMindNest
Copy link
Copy Markdown
Collaborator

@AuraMindNest AuraMindNest commented May 7, 2026

Closes #37

Summary by CodeRabbit

  • Tests
    • Added comprehensive QuickBook format tests and fixtures plus new QuickBook utility tests; updated AsciiDoc toolkit tests to cover importing translated targets and po4a-based flows.
  • Bug Fixes
    • Improved conversion/merge to preserve existing targets, normalize context matching, and apply fuzzy-state handling consistently; refined QuickBook positional merge and source/target filling behavior.
  • Chores
    • CI now ensures po4a is available for AsciiDoc-related tests.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 7, 2026

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds QuickBook parser unit tests and fixtures, QuickBookFormat integration tests, and updates QuickBook and AsciiDoc merge logic to normalize (source, context) matching and apply DB targets with fuzzy handling.

Changes

QuickBook Format Test Coverage

Layer / File(s) Summary
Test Imports and Constants
weblate/formats/tests/test_convert.py
Adds QuickBook typing imports, QuickBookFormat import, and fixture constants (cs.qbk, cs2.qbk); wraps existing_units with cast("list[Unit]", ...).
QuickBook Test Data Files
weblate/trans/tests/data/cs.qbk, weblate/trans/tests/data/cs2.qbk
Two Czech QuickBook article fixtures with headings, paragraphs, and a Weblate demo link used by parser and integration tests.
QuickBook utils: merge logic
weblate/utils/quickbook.py
Introduce module-scope fuzzy-state set and _qbk_po_unit_merge_context; normalize merge keys to (source, context), update existing PO unit targets/fuzzy flags on match, and create missing PO units on miss when merging existing_units.
QuickBook format load/save
weblate/formats/quickbook.py
Extract helpers for empty PO creation, template path resolution, conditional fill of targets when loading same-template file (respect existing_units), and positional merge for translated .qbk files.
AsciiDoc merge adjustments
weblate/formats/asciidoc.py
Import FUZZY_STATES and rewrite _merge_translations to normalize stored-unit context lookups and apply DB targets (overwrite or add) with fuzzy marking.
Parser Unit Tests
weblate/utils/tests/test_quickbook.py
Adds tests for bracket matching, bracket-keyword parsing, block/section/paragraph parsing, code-block skipping, paragraph joining, inline bracket preservation, qbk→po location checks, and po→qbk replacement/fallback behavior.
Format Integration & Typing
weblate/formats/tests/test_convert.py, ci/apt-install
Adds QuickBookFormatTest with round-trip, existing-units save, and import-existing tests; tightens typing for existing_units with cast; renames local IDML var and adds isinstance check; CI apt script adds po4a package.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • cppalliance/weblate#11: Adds QuickBook tests and fixtures that exercise QuickBookFormat and qbk_to_po/po_to_qbk conversion functions.

Suggested reviewers

  • jonathanMLDev
  • henry0816191
  • wpak-ai

Poem

🐰 A rabbit hops through tests anew,
Brackets matched and paragraphs glue,
Czech headings, links, and translated cheer,
QuickBook lines in order appear.
Hooray — the parser sings, testing's here!

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 26.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Out of Scope Changes check ❓ Inconclusive Some changes extend beyond the stated objectives: modifications to asciidoc.py and quickbook.py (refactoring merge logic, fuzzy state handling) and ci/apt-install (adding po4a dependency) are implementation improvements not explicitly required by issue #37. Clarify whether implementation refactoring (FUZZY_STATES, merge logic updates) was necessary to support the test objectives or if these are separate enhancements that should be in a different PR.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: adding tests for AsciiDoc and QuickBook formats, which aligns with the primary objectives.
Linked Issues check ✅ Passed The PR successfully implements all coding requirements from issue #37: AsciiDoc format handler tests (AsciiDocToolkitFormatTest with import_existing), QuickBook format handler tests (QuickBookFormatTest), and QuickBook parser tests (test_quickbook.py).

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@weblate/utils/tests/test_quickbook.py`:
- Around line 7-95: Replace the unittest-based classes QuickBookParserTest and
QuickBookConversionTest with module-level pytest test functions: remove "import
unittest", change each method (e.g., test_find_bracket_end_nested,
test_parse_bracket_keyword_section_with_id, test_skip_include_and_parse_heading,
test_paragraph_soft_wrap_joined, test_indented_code_block_skipped,
test_section_title_and_body, test_inline_bracket_on_wrapped_line,
test_qbk_to_po_locations, test_po_to_qbk_applies_translation,
test_po_to_qbk_fallback_untranslated) into standalone def functions, use plain
assert expressions instead of self.assertEqual/self.assertIn/self.assertTrue,
and call the same helpers (_find_bracket_end, _parse_bracket_keyword,
_parse_qbk, qbk_to_po, po_to_qbk) unchanged; ensure any test setup/assertions
that referenced self are adjusted to local variables and return values so tests
run under pytest/pytest-django.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2dacccf5-978b-4ee6-815d-7c6f05a4cfd5

📥 Commits

Reviewing files that changed from the base of the PR and between d5029e2 and 893db0c.

📒 Files selected for processing (4)
  • weblate/formats/tests/test_convert.py
  • weblate/trans/tests/data/cs.qbk
  • weblate/trans/tests/data/cs2.qbk
  • weblate/utils/tests/test_quickbook.py

Comment thread weblate/utils/tests/test_quickbook.py Outdated
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 8, 2026

Caution

Failed to replace (edit) comment. This is likely due to insufficient permissions or the comment being deleted.

Error details
{}

@cppalliance cppalliance deleted a comment from codecov Bot May 8, 2026
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
weblate/formats/tests/test_convert.py (1)

395-503: ⚡ Quick win

Add one regression case for context-sensitive and fuzzy merges.

These tests only cover empty-context, non-fuzzy units, so the new (source, context) and STATE_FUZZY branches in the QuickBook merge logic can still regress unnoticed. A fixture with duplicate source text in two contexts plus one fuzzy existing_unit would pin down the behavior this PR is changing.

As per coding guidelines, "Write unit tests for new functionality and integration tests for complex features".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@weblate/formats/tests/test_convert.py` around lines 395 - 503, Add a
regression test method to QuickBookFormatTest that covers context-sensitive and
fuzzy merges: create a temp QuickBook file containing the same source string in
two different contexts (e.g., two different [heading …] or sections), call the
QuickBookFormat constructor with existing_units containing a MockUnit
constructed with the duplicate source, the matching context for one occurrence,
a fuzzy target and STATE_FUZZY, then call storage.save() and assert that only
the occurrence with the matching context was updated with the fuzzy target and
that the saved unit preserves the fuzzy state (use the same helpers as
test_existing_units: MockUnit, STATE_FUZZY, storage.save(),
Path(...).read_text/encoding checks). Ensure the new test method is named
clearly (e.g., test_existing_units_context_and_fuzzy) and follows the same
temp-file cleanup pattern as other tests.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@weblate/formats/asciidoc.py`:
- Around line 58-69: The code only sets fuzzy on matched/created units but never
clears it, so ensure the fuzzy flag is synchronized both ways: when handling
lookups via store_units_index and when creating via store.addsourceunit (and
after thepo.setcontext/target assignment), check existing_unit.state against
STATE_FUZZY and call thepo.markfuzzy(True) if fuzzy or thepo.markfuzzy(False) if
not; update both branches that reference store_units_index, thepo,
existing_unit, STATE_FUZZY, markfuzzy, store.addsourceunit, setcontext and
target to keep the store unit’s fuzzy state identical to existing_unit.state.

In `@weblate/utils/quickbook.py`:
- Around line 863-867: The merge currently only sets fuzzy on the DB unit
(store_index -> po_unit) when ex_unit.state == STATE_FUZZY, so a stale fuzzy
flag can persist; update the block that handles existing keys (checking
store_index[key] and assigning po_unit.target) to explicitly clear the fuzzy
flag when ex_unit.state is not STATE_FUZZY by calling po_unit.markfuzzy(False)
(i.e., add an else branch to the existing if ex_unit.state == STATE_FUZZY
condition), ensuring po_unit reflects the extracted unit's fuzzy state.
- Around line 853-875: The merge creates duplicates because extracted units
store context as a developer note via unit.addnote(f"type: {seg.context}",
"developer") but the merge index keys on u.getcontext() (msgctxt), so contexts
appear empty; to fix, change the index-building logic that constructs
store_index to obtain context from the same representation used by extraction:
for each unit u use ctx = u.getcontext() or, if that is empty, read the
developer note that was set (the note string starting with "type: ") and extract
the context value, then use key = (u.source, ctx) so matching existing_units
(which use setcontext()/getcontext()) will find the right po_unit; update
references in the loop that builds store_index and keep references to
store_index, getcontext(), addnote(), setcontext(), existing_units,
store.addsourceunit, and new_unit.setcontext() in your change.

---

Nitpick comments:
In `@weblate/formats/tests/test_convert.py`:
- Around line 395-503: Add a regression test method to QuickBookFormatTest that
covers context-sensitive and fuzzy merges: create a temp QuickBook file
containing the same source string in two different contexts (e.g., two different
[heading …] or sections), call the QuickBookFormat constructor with
existing_units containing a MockUnit constructed with the duplicate source, the
matching context for one occurrence, a fuzzy target and STATE_FUZZY, then call
storage.save() and assert that only the occurrence with the matching context was
updated with the fuzzy target and that the saved unit preserves the fuzzy state
(use the same helpers as test_existing_units: MockUnit, STATE_FUZZY,
storage.save(), Path(...).read_text/encoding checks). Ensure the new test method
is named clearly (e.g., test_existing_units_context_and_fuzzy) and follows the
same temp-file cleanup pattern as other tests.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b0b4ba61-6aea-4c63-bebd-d8ad8886b65a

📥 Commits

Reviewing files that changed from the base of the PR and between 279c422 and acc5dbe.

📒 Files selected for processing (4)
  • weblate/formats/asciidoc.py
  • weblate/formats/tests/test_convert.py
  • weblate/utils/quickbook.py
  • weblate/utils/tests/test_quickbook.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • weblate/utils/tests/test_quickbook.py

Comment thread weblate/formats/asciidoc.py Outdated
Comment thread weblate/utils/quickbook.py
Comment thread weblate/utils/quickbook.py Outdated
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
weblate/formats/quickbook.py (1)

48-135: ⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

CI blocker: convertfile exceeds the project's complexity budget (ruff C901: 17 > 16).

The pre-commit pipeline is failing on this method. The new if self.existing_units / else branch added at lines 95–102 pushed the cyclomatic complexity above the configured threshold. Extract one of the two large blocks (e.g., the storefile_path == template_path source-language fill, or the translated-file positional-import else branch) into a private helper.

♻️ Suggested refactor (extract the source-language fill)
+    def _fill_source_language_targets(self, store: TranslationStore) -> None:
+        """When loading the source-language file, populate empty unit targets with source."""
+        merged = bool(self.existing_units)
+        for unit in store.units:
+            if unit.isheader():
+                continue
+            if merged:
+                if not unit.target:
+                    unit.target = unit.source
+            else:
+                unit.target = unit.source
+
     def convertfile(
         self,
         storefile: IO[bytes],
         template_store: TranslationFormat | None,
     ) -> TranslationStore:
         ...
         storefile_path: str | None = getattr(storefile, "name", None)
         if storefile_path == template_path:
-            for unit in store.units:
-                if unit.isheader():
-                    continue
-                if self.existing_units:
-                    if not unit.target:
-                        unit.target = unit.source
-                else:
-                    unit.target = unit.source
+            self._fill_source_language_targets(store)
         elif storefile_path is None:
             ...

Alternatively, lift the translated-.qbk positional-import branch (lines 110–133) into _load_translated_qbk(storefile_path, store) — that block is self-contained and would reduce branches further.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@weblate/formats/quickbook.py` around lines 48 - 135, The method convertfile
is over the complexity limit; extract the translated-file positional-import
branch (the elif/else block that reads translated_content, builds
translated_store, compares counts and assigns targets) into a private helper
method named _load_translated_qbk(self, storefile_path: str, store:
TranslationStore) -> None; move the try/except and all logic that reads
Path(storefile_path).read_text, calls qbk_to_po, computes
trans_units/tmpl_units, performs the length check/report_error, and assigns
tmpl_unit.target from trans_unit.source into that helper, then replace the
original else branch in convertfile with a single call to
self._load_translated_qbk(storefile_path, store) so cyclomatic complexity drops.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@weblate/utils/quickbook.py`:
- Around line 870-895: The merge asymmetry comes from computing store keys with
_qbk_po_unit_merge_context(u) (which reads developer notes) while lookup uses
ex_unit.context directly; fix by making the lookup-side compute ctx from the
Weblate Unit's available fields (use ex_unit.context or, when empty, the Unit's
note/developer-note field exactly as the extractor stores it) before forming
key=(src, ctx), so existing_units matching uses the same context logic as
store_index; alternatively, standardize store-side to use u.getcontext() and
ensure the extractor calls setcontext(ctx) during extraction, but do not call
translate-toolkit-only methods on Weblate Unit objects (avoid
getcontext()/getnotes() on ex_unit).

---

Outside diff comments:
In `@weblate/formats/quickbook.py`:
- Around line 48-135: The method convertfile is over the complexity limit;
extract the translated-file positional-import branch (the elif/else block that
reads translated_content, builds translated_store, compares counts and assigns
targets) into a private helper method named _load_translated_qbk(self,
storefile_path: str, store: TranslationStore) -> None; move the try/except and
all logic that reads Path(storefile_path).read_text, calls qbk_to_po, computes
trans_units/tmpl_units, performs the length check/report_error, and assigns
tmpl_unit.target from trans_unit.source into that helper, then replace the
original else branch in convertfile with a single call to
self._load_translated_qbk(storefile_path, store) so cyclomatic complexity drops.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 91c8f4a0-65ff-462a-8697-ef112c89f1c2

📥 Commits

Reviewing files that changed from the base of the PR and between acc5dbe and 6c39b4e.

📒 Files selected for processing (4)
  • weblate/formats/asciidoc.py
  • weblate/formats/quickbook.py
  • weblate/formats/tests/test_convert.py
  • weblate/utils/quickbook.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • weblate/formats/tests/test_convert.py

Comment thread weblate/utils/quickbook.py
@AuraMindNest AuraMindNest requested a review from jonathanMLDev May 8, 2026 16:13
@codecov
Copy link
Copy Markdown

codecov Bot commented May 8, 2026

⚠️ JUnit XML file not found

The CLI was unable to find any JUnit XML files to upload.
For more help, visit our troubleshooting guide.

@wpak-ai wpak-ai merged commit a46e1af into cppalliance:develop May 8, 2026
42 of 50 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add some tests for Asciidoc and Quickbook format.

3 participants