Skip to content

fix: lstrip_blocks, strftime_now, loop controls, caller() and filters on none - #26

Merged
leehack merged 4 commits into
mainfrom
fix/lstrip-set-and-strftime
Sep 25, 2026
Merged

leehack merged 4 commits into
mainfrom
fix/lstrip-set-and-strftime

Conversation

@leehack

@leehack leehack commented Sep 25, 2026 •

Copy link
Copy Markdown
Owner

Closes #24
Closes #25
Closes #27
Closes #28
Closes #29

Five parity fixes. Reference: llama.cpp 7fe450e1 test-jinja, built locally with a render hook. Jinja2 3.1.6 runs as SandboxedEnvironment(trim_blocks=True, lstrip_blocks=True, loopcontrols), llama.cpp's -py setup. Where the two disagree, dinja follows llama.cpp, except for one llama.cpp bug, noted below.

What changed

  • lstrip_blocks (lstrip_blocks keeps indentation before a tag on the line after a block tag #24): the lexer decided whether a {% or {# starts a line after trim_blocks had already removed the text's leading newline. Its fallback, the character before the text, is always the previous tag's }, so the indentation was never removed. The check now runs on the raw text first, as in llama.cpp's lexer.cpp, for every tag and comment.
  • strftime_now (strftime_now ignores most strftime conversions (%b, %B, %a, %c, %%, ...) #25): new lib/src/runtime/strftime.dart, a port of glibc 2.41 strftime in the C locale, which llama.cpp calls. It covers every conversion, the E/O modifiers, flags and widths. As in llama.cpp, it throws without a string argument and on an empty or 100+ byte result. Before, strftime_now() fell back to a default format. The README now says it reads the system clock in the local time zone; values passed to render (such as llamadart's now) don't change it.
  • continue/break (continue and break drop the output of the current loop pass #27): the signal now carries the output the pass rendered before it. A for's else runs unless some pass finishes without continue or break, as in both engines.
    • One exception: output rendered in a nested block before the signal ({% if %}x{% continue %}{% endif %}) is kept, as in Jinja2. llama.cpp drops the output of the nested blocks, which I count as a llama.cpp bug. No template in either corpus renders anything before a signal in a nested block.
  • Audit fixes at b3eaffc:
    • A break or continue that leaves a captured body (block set, filter block, macro, caller() body) now discards the captured text, as in llama.cpp and Jinja2. At 2d60d4d it leaked into the loop output.
    • Output before the capture is still kept: b{% set s %}a{% continue %}{% endset %} in a loop gives bb, as in both.
    • For a macro or caller() body, Jinja2 rejects continue outside a loop, so this follows llama.cpp alone.
    • A break/continue in a for's else block now keeps the loop's earlier output. For {% for i in [1, 2] %}{% for j in [1] %}a{% continue %}{% else %}x{% continue %}{% endfor %}|{% endfor %}, dinja gives Jinja2's axax. llama.cpp gives '', its nested-block bug, and base gave ||.
    • for output is now marked safe, so input-marked values in the loop body or its else block are escaped once. Base already double-escaped the body.
  • caller() (caller() renders None for a body statement without output #28): renders its body with execStatements, as a macro body does. Statements without output add nothing, and JinjaString.user values are now escaped. Before, they were not escaped.
  • Filters on none (none | selectattr(...) | list gives [None] instead of [] #29): select, reject, selectattr, rejectattr and unique now return [] for none. map and items already did. This matches llama.cpp's value_none_t::get_builtins. default now also replaces none, as llama.cpp's default_value does (Jinja2 keeps none).
  • README: the real-templates bullet now states the 86-template parity below, and a scope bullet covers strftime_now and the system clock.
  • CHANGELOG: one bullet per fix under ## Unreleased. origin/main (docs: lead README and pubspec with chat template rendering #23) is merged in, not rebased.

Not changed: none | list, join, length, count, first, last, sort, reverse, sum, min, max and dictsort. llama.cpp and Jinja2 both raise for these; dinja still returns a value ([None], None, 0, "" or []). Making them raise would turn renders that work today into errors, so I left them for a separate decision.

Parity with llama.cpp 7fe450e1

main (74d487e) #26 first push (97626f9) #26 now
86 distinct templates (45 fixtures + llama.cpp models/templates) × 4 shapes: same / both error / differ 302 / 7 / 35 (14 templates) 333 / 7 / 4 (2) 337 / 7 / 0 (unchanged at b3eaffc)
2,550 whitespace probes matching 2,411 2,489 2,538 (unchanged at b3eaffc)
  • Shapes: system prompt with tools and a generation prompt; user only; multi-turn; tool call and result.
  • Both error: the 7 shared errors are the templates' own raise_exception for shapes they reject (gemma-2 system role, gemma alternation, Mistral tool-call IDs).
  • Probes: the 12 still differing are the llama.cpp first-text bug. a{% if true %}b{% endif %} renders b in llama.cpp; dinja keeps Jinja2's ab, pinned by a test.
  • Printing: {{ none | selectattr("a") }} (printed without | list) now shows []; llama.cpp prints any empty list as "". This follows dinja's existing Jinja2-style list printing ({{ [1, 2] }} is [1, 2], while llama.cpp gives 12), which this PR doesn't change.

strftime: 9,382 formats × 18 timestamps × 5 time zones gives 844,380 cases against glibc 2.41 (Docker gcc:14), with 0 differences. %Z is DateTime.timeZoneName, whose format depends on the platform. On the web it is the browser's long name (Eastern Daylight Time, where llama.cpp gives EDT), as the README and Dartdoc now say. No template uses %Z.

Tests

Every expected value was checked against llama.cpp and Jinja2 (or glibc for strftime), except where a comment names the one engine that gives it.

Fix New tests failing on main Guard tests passing on both Mutation → failing tests
lstrip_blocks 12 12 trim before lstrip → 12
strftime_now 7 (plus 84 strftime cases for the new function) — %b gives the full name → 20
continue/break 11 3 drop the carried output → 11; count a continued pass as iterated → 7
caller() 7 2 old string concatenation → 7
Filters on none 8 8 selectattr passes none through → 3; default keeps none → 1
Captured output on break/continue (audit B1) and else-block signals 13 fail at 2d60d4d 3 keep captured text → 11; drop loop output before an else signal → 2
Loop output escaped once (audit B2) 7 fail at 2d60d4d — loop output not marked safe → 7

llamadart consumer check

llamadart main (c71de5976), with dependency_overrides to dinja main (74d487e), 2d60d4d and b3eaffc. b3eaffc and 2d60d4d give identical results: the same test results and 0 of 1,096 renders differ. Against main:

Gates

dart format clean · dart analyze no issues · dart test +1069 ~11 · Chrome JS and Wasm (test/src, cross tests) +967 ~11 each · dart pub publish --dry-run 0 warnings · pana 160/160 · README and example snippets run with the stated output.

@leehack leehack changed the title fix: lstrip_blocks after a trimmed newline and full strftime_now fix: lstrip_blocks, strftime_now, loop controls, caller() and filters on none Sep 25, 2026
…re an else-block signal, escape loop output once
@leehack
leehack merged commit 3003bcf into main Sep 25, 2026
1 check passed
@leehack leehack mentioned this pull request Sep 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment