Skip to content

fix: macro kwargs, unary operators, min/max attribute, indent, format filter and llama.cpp 7fe450e1 parity - #20

Merged
leehack merged 8 commits into
mainfrom
fix/macro-kwargs-unary-parity
Sep 25, 2026
Merged

leehack merged 8 commits into
mainfrom
fix/macro-kwargs-unary-parity

Conversation

@leehack

@leehack leehack commented Sep 25, 2026 •

Copy link
Copy Markdown
Owner

Closes #14, closes #15, closes #16, closes #17, closes #18, closes #21, closes #22.

Causes and fixes

  • Macro keyword arguments in a different order render empty values #14 macro arguments. Binding read positional slots only and consulted keywords only for parameters with defaults. Now: positional, then keyword by name, then default. A parameter left unfilled is undefined while arguments remain for its position, then Not enough arguments provided to 'f' (llama.cpp's rule and message). The same binding now applies to caller(...) parameters of a {% call %} block (keywords, defaults and the missing-argument error were ignored before).
  • Unary minus/plus on a variable fails to parse (including slice bounds) #15 unary -/+. The lexer already emitted unary tokens, but the parser had no rule for them. Added llama.cpp's parse_unary_expression between filters and call/member, so -n|abs is (-n)|abs, -n is number is (-n) is number, -a ** 2 is (-a) ** 2 (Jinja2's precedence). The lexer also treats -/+ after the not operator as unary (not after .not or |not).
  • min/max(attribute=) return the attribute instead of the item #16 min/max(attribute=). Returned the attribute value; now return the item, and read attribute positionally too.
  • Remaining parity gaps with llama.cpp 7fe450e1's Jinja tests #17. Numeric dot access evaluates as a subscript and rejects negatives; a[] parses to a new BlankExpression (exported from ast.dart) that is undefined; int * str repeats; indent accepts a string width; str.format supports {} fields with llama.cpp's input marking. Other fields ({0}, {name}, {{, {:>5}) throw format() only supports simple '{}' placeholders, as llama.cpp does.
  • Generator. Cases llama.cpp 7fe450e1 marks not-implemented are emitted with skip:. Adjacent C++ string literals are now joined correctly; before, 4 multi-line templates kept " " junk.
  • Lexer deletes literal " " (8 spaces in quotes) from template text #21. The lexer deleted every " " from template text, a workaround for the generator bug above. Removed. Nothing else depended on it: the only other occurrences (two Hermes fixtures) are inside {{ }} string literals, which the workaround never touched.
  • Parity gaps: indent trailing newline, unary minus in test args, call splat, format filter #22 (1) indent. Rewritten on llama.cpp's line model: a trailing newline is kept once (plus the indent with blank=true), so 'foo\n'|indent is foo\n. Two exceptions. llama.cpp drops leading empty lines when first=false ('\nfoo'|indent is foo, an obvious bug), so those follow Jinja2 (\n foo), as dinja already did. ''|indent(2, true) is '' as in llama.cpp (Jinja2 gives ' ').
  • Parity gaps: indent trailing newline, unary minus in test args, call splat, format filter #22 (2) test arguments. a is divisibleby -a already parses as (a is divisibleby) - a, as it does in llama.cpp and Jinja2. Neither engine accepts - as the start of an unparenthesized test argument. Both then throw, because divisibleby got no argument; dinja returned False and rendered -2. The issue's expected True is not what either engine gives. Tests that take an argument (divisibleby, eq/equalto, ne, gt/greaterthan, ge, lt/lessthan, in, plus dinja's le, sameas, startingwith, endingwith, ieq) now throw Test expected 2 arguments, got 1 without one, including through select('eq').
  • Parity gaps: indent trailing newline, unary minus in test args, call splat, format filter #22 (3) * unpacking. llama.cpp parses f(*x) but has no runtime for it (cannot exec SpreadExpression). Following that, dinja now throws Argument unpacking with * is not supported instead of passing the list as one argument. Jinja2 would unpack.
  • Parity gaps: indent trailing newline, unary minus in test args, call splat, format filter #22 (4) format filter. In llama.cpp, '...'|format(...) reaches the same {}-only string format builtin, so dinja's new filter calls str.format: '{}-{}'|format(1, 2) is 1-2 and '%s-%s'|format(1, 2) is %s-%s, as in llama.cpp. Non-strings throw Unknown filter 'format' for type Integer, as llama.cpp does. Jinja2's %-formatting ('%s-%s'|format(1, 2) gives 1-2) is not implemented.
  • README input-marking example claims plain strings are escaped #18. README and example/security_example.dart now wrap user input in JinjaString.user, and say that plain strings are not escaped. Example output:
    1. Plain string (not escaped):
    Hello <script>alert("xss")</script>!
    2. User input (auto-escaped):
    Hello &lt;script&gt;alert(&quot;xss&quot;)&lt;/script&gt;!
    3. User input marked as safe (raw html):
    Hello <script>alert("xss")</script>!
    

Decisions where llama.cpp and Jinja2 differ

Case llama.cpp 7fe450e1 Jinja2 3.1.6 This PR
f() for f(a, b=2) Not enough arguments [|2] throws (llama.cpp)
f(1, z=3) [1|] (count bug) TypeError throws takes no keyword argument 'z'
f(1, a=2) [2|] TypeError throws got multiple values for argument 'a'
f(b=5) [|2] (drops b=5) [|5] [|5]
f(1, 2, 3) [1|2] TypeError [1|2] (unchanged)
not -n, -a ** 2, -b // a parse error (lexer, no **///) False, 4, -3 Jinja2
n.1 on an int error '' '' (same as n[1])
'<{}>'.format(undefined) error <> <>
'a}b'.format() a}b ValueError a}b
'\nfoo'|indent foo (drops the line) \n foo Jinja2
''|indent(2, true) '' ' ' llama.cpp
f(*[1, 2, 3]) cannot exec SpreadExpression 1|2|3 throws (llama.cpp)
'%s-%s'|format(1, 2) %s-%s 1-2 llama.cpp

The #14 brief asked for errors on unknown and duplicate keywords "the way llama.cpp does". llama.cpp doesn't raise those errors; it binds them wrong. This PR throws as Jinja2 does. varargs/kwargs are still unsupported, as in llama.cpp. A macro whose body uses kwargs and is called with extra keywords now throws; before, it rendered kwargs as empty. No fixture does this. min/max stay case-sensitive, as in llama.cpp. dinja has no case_sensitive support, so Jinja2's case-insensitive default isn't matched.

Evidence

llama.cpp outputs come from its common/jinja built standalone at 7fe450e1. Jinja2 3.1.6 outputs come from a SandboxedEnvironment with the same options as test-jinja.cpp -py. Each of the 175 expectations in the new tests matches the engine its comment names.

Tests

179 new cases. 146 fail on main (3ff742e) and pass here. 33 are labelled guards ("unchanged", "Binary minus is unchanged", "still throws", "stays binary") and pass on both. The guards cover binary minus, b - -k, b - -2, negative literals, (k) -b, lst[1] -k, k -}}, d.not - 1 and d|not - 1, all with unchanged trees. Two existing tests that asserted min/max(attribute=) returning the value now assert the item.

Mutations (all killed unless noted)

# Mutation Killed by
M1 filter operand skips unary 41 tests
M2 unary wraps the filter (-(n|abs)) -m|abs, tree test
M3 a[] not blank 9
M4 - after not binary 4
M5 - after .not/|not unary 2
M6 numeric dot removed 12
M7 negative dot index allowed 1
M8 int * str removed 6
M9 int * str drops input marking 1
M10 keywords ignored in binding 10
M11 unknown keyword accepted 1
M12 duplicate keyword accepted 2
M13 unfilled parameter always throws 1
M14 missing argument never throws 4
M15 caller counted as an argument 1
M16 caller rejected as unknown keyword 10
M17 caller(...) parameters unbound 6
M18/M19 min/max return the value 9 / 5
M20 positional attribute ignored 1
M21 max tie keeps the last item 1
M22 string indent width ignored 6
M23 format removed 16
M24/M25 format drops literal / argument marking 1 / 1
M26 format accepts other fields 5
M27 missing-argument check removed 1
M28 BlankExpression evaluates to none survives, equivalent: member access maps none and undefined subscripts alike
M29 BlankExpression evaluates to 0 2
G1 generator skip list ignored 3 cross tests
G2 generator's old literal joining 4 cross tests (after #21)
M30 lexer strips " " again 3
M31 indent keeps the trailing empty line 11
M32 test argument not required 18
M33 * passes the value through 5
M34 format filter removed 8
M35 format filter stringifies non-strings 2

Cross-test regeneration

test/llama_cross_test.dart is regenerated from 7fe450e1's tests/test-jinja.cpp. The old file had 243 cases; the new one has 290. The generated names, templates, data, expectations and skip flags match, for all 290, the cases dumped from upstream's compiled test-jinja (279 pass, 11 skip, 0 fail).

Consumer check (llamadart c0dbdc1d6, throwaway copy, dependency_overrides only)

Current llamadart origin/main is 269d337f4. It differs from c0dbdc1d6 in lib, test and pubspec.yaml only by doc comments and a CI script path.

  • test/unit/core/template and test/integration/core/template: identical with dinja main and with this branch: 757 pass, 74 skip, 2 fail. The 2 failures (tool_call_fixture_render_test: Llama 3.2, Ministral 3) also fail on main. They come from 1.1.1's tojson spacing; llamadart pins <1.1.0.
  • Render diff: 58 templates (llamadart's 18 fixtures plus dinja's), 7 ChatTemplateEngine scenarios each plus JinjaAnalyzer caps: 464 results, 0 differences. A static scan of 185 macro calls in those templates finds none that the new binding errors reject.

Gates

dart format clean, dart analyze no issues, dart test 892 pass / 11 skip, coverage 90.17%, dart test -p chrome test/src test/llama_cross_test.dart 790 pass / 11 skip, dart pub publish --dry-run 0 warnings, pana 160/160.

Unrelated, not fixed (llama.cpp differs, dinja matches Jinja2)

  • {{ [1, 2]|select('eq', 2)|list }}: llama.cpp prints 2, dinja and Jinja2 print [2].
  • {{ '{}'.format(x) }} with x undefined: llama.cpp throws Undefined (hint: 'x') is not a string value, dinja and Jinja2 print nothing.
  • {{ n.1 }} on an integer: llama.cpp throws Cannot access property with non-string: got Integer, dinja and Jinja2 print nothing.

@leehack leehack changed the title fix: macro keyword arguments, unary operators, min/max attribute and llama.cpp 7fe450e1 parity fix: macro kwargs, unary operators, min/max attribute, indent, format filter and llama.cpp 7fe450e1 parity Sep 25, 2026
@leehack
leehack merged commit f682855 into main Sep 25, 2026
1 check passed
@leehack leehack mentioned this pull request Sep 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment