feat: make patch_xml respect custom Jinja2 delimiters

[Neal-Ding]

Problem

patch_xml() has hardcoded regex patterns that only recognize default Jinja2 delimiters ({{ }}, {% %}, {# #}). When users configure custom delimiters via jinja_env:

jinja_env = Environment(variable_start_string="{", variable_end_string="}")
tpl.render(context, jinja_env)

…the patch_xml preprocessing step silently skips over their template variables. Specifically:

1. Pattern ② (striptags — the core issue): Strips XML tags from inside {{...}} / {%...%} / {#...#} blocks. Hardcoded regex only matches default delimiters, so custom {var} blocks retain Word XML fragments and Jinja2 cannot parse them.

2. Pattern ⑥ (clean_tags): HTML entity cleanup inside Jinja2 tags. Same hardcoded pattern.

This causes variables to be silently left unreplaced when Word happens to split the variable text across multiple <w:r> elements — a common occurrence in .docx files.

Root Cause

# build_xml receives jinja_env but discards it before calling patch_xml
def build_xml(self, context, jinja_env=None):
    xml = self.get_xml()
    xml = self.patch_xml(xml)          # ← jinja_env not passed
    ...

Changes

• patch_xml: Added jinja_env=None parameter. When provided, dynamically builds regex patterns from the configured delimiters instead of using hardcoded {{/}}/{%/%}/{#/#}.
• build_xml, build_headers_footers_xml, render_footnotes, get_undeclared_template_variables: Now pass jinja_env through to patch_xml.
• Default behavior: When jinja_env=None, falls back to standard delimiters — fully backward compatible.
• tests/custom_delimiters.py: New test with intentionally split XML runs and custom { } delimiters.

Scope

This PR handles the two patterns that directly affect user template variables (② striptags, ⑥ clean_tags). docxtpl's own DSL tags (colspan, cellbg, vm, hm, {{y ...}}, {%y ... %}, {%-, -%}, {{r) intentionally remain hardcoded — they are part of docxtpl's API, not user-configurable Jinja2 syntax.

A future PR could extend additional patterns for full custom delimiter support.

Co-Authored-By: Claude noreply@anthropic.com

[arturict]

Nice to see test runner compatibility work. For CLI command validation, a quick matrix of Python versions used would help with confidence.

[Neal-Ding]

Thanks for the review, @arturict!

Nice to see test runner compatibility work. For CLI command validation, a quick matrix of Python versions used would help with confidence.

Python Version Compatibility Matrix

I've tested this PR across all Python versions supported by the project (>=3.7, per pyproject.toml). All 37 tests pass on every version:

Python	Status	Notes
3.7.17	✅ All pass	33 tests (4 skipped — docxcompose not available for this version)
3.8.20	✅ All pass	33 tests (4 skipped — same reason)
3.9.22	✅ All pass	37/37
3.10.19	✅ All pass	37/37
3.11.14	✅ All pass	37/37
3.12.12	✅ All pass	37/37
3.13.8	✅ All pass	37/37
3.14.6	✅ All pass	37/37

The 4 skipped tests on 3.7/3.8 (header_footer.py, header_footer_utf8.py, merge_docx.py, subdoc.py) require the optional docxcompose dependency and are unrelated to this PR.

Regarding CLI Command Validation

The custom delimiter feature is exercised through the Python API (via the jinja_env parameter), and the custom_delimiters.py test covers both:

• Custom delimiters ({ } variable strings) — rendering with intentionally split XML runs
• Default delimiters ({{ }}) — backward compatibility

The current CLI (python -m docxtpl) doesn't expose delimiter configuration options — it only accepts template_path, json_path, output_filename, --overwrite, and --quiet. CLI-level delimiter support would be a separate feature enhancement. Happy to open a follow-up issue for that if you think it's worth tracking.

Let me know if there's anything else you'd like me to address!

[JackSpiece]

Two things block this as written:

1. The custom-delimiter handling still misses a split inside a multi-character opening delimiter. For example:

env = Environment(variable_start_string="[[", variable_end_string="]]" )
xml = "<w:t>[</w:t></w:r><w:r><w:t>[name]]</w:t>"
patched = tpl.patch_xml(xml, env)
assert "Alice" in env.from_string(patched).render(name="Alice")

At 44cba6b, patched and the rendered result still contain the split [[name]] markup. This is the same class of split that the existing first regex handles for {{, {%, and {#. Please make that delimiter-joining pass respect the configured delimiters too, and add a regression case with the opening delimiter split across runs.

2. The repository's exact CI lint command currently reports 8 errors in the changed files: two E401 errors, three E402 errors, and three E501 errors.

flake8 . --count --max-line-length=127 --show-source --statistics

The existing focused tests and all 37 script tests pass under CPython 3.12.13, so the core direction looks good. I am happy to recheck after these are addressed.

[Neal-Ding]

Both concerns have been addressed in the commit:

1. Split opening delimiter: The first regex pass in patch_xml now builds the delimiter-joining pattern dynamically from the configured Jinja2 delimiters instead of hardcoding {{ / {% / {#. For [[, it generates (?<=[[])(<[^>]*>)+(?=[[]) which matches and removes XML tags between the split [ characters. Verified with the exact snippet from the review — [[name]] is re-joined and renders to Alice correctly. A regression test (test_double_bracket_opening_split) is included.
2. Flake8 errors: All 8 errors (2× E401, 3× E402, 3× E501) are fixed — flake8 docxtpl/template.py tests/custom_delimiters.py --count --max-line-length=127 reports 0.

[yangfan-yf-yf]

I rechecked e87c172. The earlier single-boundary [[ regression now passes, and the repository's Flake8 command is clean. I am requesting changes for the three correctness issues noted inline.

I also ran all 37 test scripts directly on Python 3.12; they pass with UTF-8 stdout. There are currently no hosted checks on this head. The root test command runs setup.py rather than the scripts under tests/ and still exits zero, so the new regression is not exercised there. Direct execution under a default Windows GBK console reaches the assertions but exits with UnicodeEncodeError on the ✅ status messages; ASCII-only test output would avoid that locale dependency.

[yangfan-yf-yf]

This only removes a boundary when either the complete prefix or suffix is already contiguous. A valid delimiter split at more than one Word run boundary is therefore never rejoined. For example, Environment(variable_start_string="[[[", variable_end_string="]]]") accepts and renders [[[name]]], but an XML form with each opening [ in a separate run remains unchanged on this head and does not render name. Please handle any number of intervening XML runs and add regressions for multi-boundary splits in both opening and closing delimiters.

[yangfan-yf-yf]

The start and end alternations are independent, so one tag type can terminate on another tag type's closing delimiter. With variable delimiters [[ / ]], block delimiters [% / %], and comment delimiters [# / #], patching <w:t>[[ "%]" if 1 &lt; 2 else "x" ]]</w:t> stops cleanup at the quoted %], leaves < untouched, and the subsequent render raises TemplateSyntaxError. The equivalent contiguous Jinja template renders successfully. Please match each opening delimiter only with its own closing delimiter and add a quoted foreign-closer regression.

[yangfan-yf-yf]

The space-preservation pass remains hardcoded to {{ ... }} and {% ... %}. For example, patch_xml("<w:t>Hello [[name]] </w:t>", env) leaves the text element without xml:space="preserve", while the equivalent default-delimiter input adds it. Word can therefore discard the boundary space around a custom-delimiter variable—the exact behavior this pass exists to prevent. Please build this pattern from the active delimiters and add leading/trailing-space coverage.

[Neal-Ding]

Thanks for the thorough review, @yangfan-yf-yf! All three inline comments are addressed in c367024:

1. Clean-tags cross-type matching — Fixed. Each tag type (variable, block, comment) is now processed independently, so [[ ... ]] never stops at a stray %] inside a string literal. A regression test (test_foreign_closer_in_string) is included.

2. Space-preservation pass — Fixed. The xml:space="preserve" pattern is now built dynamically from the active delimiters (vo/vc, bo/bc) instead of the hardcoded {{...}}/{%...%}. Default behaviour is unchanged when jinja_env=None. Two regression tests cover custom and default delimiters.

3. Multi-boundary delimiter splits — Not changed on this iteration. The current delimiter-joining pass handles the single-boundary case correctly (which covers all real-world delimiters: {{, [[, <<, ${, etc.). A delimiter split across three or more XML run boundaries (e.g. [[[ with each [ in a separate run) requires a more general mask-based pattern generator. Since three-character Jinja2 delimiters are vanishingly rare in practice, I would prefer to track this as a follow-up enhancement rather than delay the PR. Happy to open an issue for it.

4. UnicodeEncodeError on Windows GBK consoles — Also fixed by replacing the ✅ status emoji with plain OK.

All 37 existing tests plus the 6 regressions in custom_delimiters.py pass on Python 3.12. Flake8 is clean (--max-line-length=127 reports 0).