fix: apply the template extensions in the job parameter merge - #368
Merged
leongdl merged 1 commit intoSep 17, 2026
Conversation
preprocess_job_parameters re-validates each merged job parameter definition
through parse_model, and it passed no supported_extensions. The parsing context
that reached NameIdentifierLengthMixin therefore carried an empty extension set,
so the mixin enforced the base 64-character limit on a parameter name that decode
had already accepted at up to 512 under FEATURE_BUNDLE_1. create_job calls
preprocess_job_parameters itself, so both entry points rejected the name.
Measured before the change, for a template declaring FEATURE_BUNDLE_1 with a
512-character STRING parameter name:
decode_job_template OK
preprocess_job_parameters DecodeValidationError: 1 validation errors for
JobStringParameterDefinition name: name must be
at most 64 characters long
create_job same
merge_job_parameter_definitions now derives the union of the extensions declared
by the job template and any environment templates, and threads it to
merge_job_parameter_definitions_for_one and on into parse_model. The union is
sound because a definition was decoded under its own template's extension set and
every definition of one job parameter carries the same name, so the union cannot
admit a value decode did not already accept for the template that declared it.
The public signature of merge_job_parameter_definitions is unchanged: it already
receives the templates that carry the extensions, so the set is derived rather
than passed. This follows the shape of OpenJobDescription#362, which fixed the sibling defect in
instantiate_model, and it keeps a caller from declaring an extension the template
did not. merge_job_parameter_definitions_for_one takes a new keyword-only
supported_extensions argument that defaults to None, preserving its behaviour for
any existing caller.
Tests cover all four FEATURE_BUNDLE_1 length ceilings at job-creation time on
both lanes. v0 gains group H in test_feature_bundle_1.py: the parameter name
through the merge, preprocess_job_parameters and create_job for each of STRING,
PATH, INT and FLOAT, a genuine two-source merge, the environment-template-only
case, the EXPR-typed model_copy branch, and controls for the base 64 limit and
the static 512 identifier ceiling. v1 gains an equivalent class in
model_v1/test_create_job.py, which had no coverage of these ceilings at all; the
Rust implementation validates in a single pass and already passes, so those tests
are a ratchet against a re-validation pass being added upstream unnoticed.
Every v0 test was mutation-checked. Seven mutants, each caught: dropping the
threading at any of its three points, hardcoding FEATURE_BUNDLE_1 instead of
deriving it, ignoring environment templates, applying the threading only to
STRING, and applying it only to single-source merges.
Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>
AlexTranAmz
approved these changes
Sep 17, 2026
leongdl
commented
Sep 17, 2026
jericht
approved these changes
Sep 17, 2026
wyongzhi
approved these changes
Sep 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
preprocess_job_parametersre-validates each merged job parameter definition throughparse_model, and it passed nosupported_extensions. The parsing context that reachedNameIdentifierLengthMixintherefore carried an empty extension set, so the mixin enforced the base 64-character limit on a parameter name thatdecode_job_templatehad already accepted at up to 512 under FEATURE_BUNDLE_1.create_jobcallspreprocess_job_parametersitself, so both entry points rejected the name.merge_job_parameter_definitionsnow derives the union of the extensions declared by the job template and any environment templates, and threads it down to theparse_modelcall.This is the sibling of the defect #362 fixed in
instantiate_model. That PR closed three of the four FEATURE_BUNDLE_1 length ceilings; this one closes the fourth, which #362's tests did not reach because their templates declare noparameterDefinitions.Where this sits in the parse flow
openjd-rsnumbers the stages a template goes through, and the Python implementation performs the same work in the same order, so the numbering is a useful common vocabulary. Passes 1 to 4 are decode: text, to a generic mapping, to typed models. Passes 5 to 10 are template validation, enumerated in the module doc ofvalidate_v2023_09/mod.rs— limits, structure, FEATURE_BUNDLE_1, format strings, TASK_CHUNKING, WRAP_ACTIONS. Job creation is the two stages after those: parameter preprocessing, then instantiation. Session execution follows.The four raised length ceilings are a pass 5 rule. They are not a property of a value on its own; they are a property of a value together with the declared extension set. In
openjd-rsthat pairing is explicit —EffectiveLimits::from_contextreads the extensions and yieldsmax_identifier_len: if fb1 { 512 } else { 64 }, andlimits::enforce_limitsapplies it once. In Python the same rule lives in the field validators, which read the extension set from theModelParsingContextthatparse_modelbuilds per call:Where it broke
Python re-runs those validators after pass 5, because pydantic validates whenever a model is constructed, and both job-creation stages construct models. That is the whole defect class: a re-validation that does not carry the context silently downgrades to the base limit and rejects a value pass 5 accepted.
flowchart TD D["Passes 1 to 4: decode, decode_job_template"] V["Passes 5 to 10: template validation. Ceilings applied here, context present"] P["Parameter preprocessing: preprocess_job_parameters"] M["merge_job_parameter_definitions_for_one calls parse_model, which re-validates"] I["Instantiation: create_job calls instantiate_model, which re-validates"] J["Job"] E1["Context dropped, so the parameter name is rejected at 64. This PR."] E2["Context dropped, so job name, environment name and filename are rejected at their base limits. Fixed by PR 362."] D --> V V --> P P --> M M --> I I --> J M --> E1 I --> E2Two sites, one cause:
_internal/_create_job.py:295target_model.model_validate(fields, context=context)target_model(**fields)_merge_job_parameter.py:256parse_model(..., supported_extensions=...)parse_model(model=..., obj=...)The call chain to the second one, which is why both public entry points fail:
preprocess_job_parameters(_create_job.py:328) →merge_job_parameter_definitions(:406) →merge_job_parameter_definitions_for_one→parse_modelcreate_job(_create_job.py:569) →_create_job_and_symbol_table(:462) →preprocess_job_parameters(:496) → the same chainWhy openjd-rs does not have it
It validates once and never re-validates a constructed model; the later stages consume resolved values instead.
create_job/parameters.rsnever constructsEffectiveLimitsat all, so there is no second computation of the ceiling to get wrong. The divergence is structural, not a missing check.Why the fix belongs here
The rule is correct and pass 5 already enforces it. What was lost is the second half of the pair — the extension set — at the one site that dropped it. So the fix restores the context at that site and changes nothing else. The alternatives are each wrong in a way worth recording:
context is None. Smaller diff, and it would accept a 512-character name on a template with noextensionsblock, because a context-free re-validation is indistinguishable from a caller that declared the extension. fix: validate instantiated job models with the template's extension context #362 rejected this same shape.Identifieralready permits 512. The ceiling is not the constraint; the conditional falling back is.supported_extensionsargument to the publicmerge_job_parameter_definitions. It already receives the templates that carryextensions, so this would ask callers to restate something derivable, and let them assert an extension the template never declared.Measured before the change
Template declaring
extensions: [FEATURE_BUNDLE_1]with a 512-character STRING parameter name:decode_job_templatemerge_job_parameter_definitionsDecodeValidationError: name must be at most 64 characters longpreprocess_job_parameterscreate_jobAfter: all four stages accept it, and
job.parameterscarries the 512-character key.Why derive rather than add an argument
merge_job_parameter_definitionsalready receives the templates that carryextensions, so no public signature changes.The union across templates is sound because a definition was decoded under its own template's extension set, and every definition of one job parameter carries the same name, so the union cannot admit a value decode did not already accept for the template that declared it.
merge_job_parameter_definitions_for_onegains a keyword-onlysupported_extensionsdefaulting toNone, which preserves its behaviour for any existing caller.Tests, both lanes
All four FEATURE_BUNDLE_1 length ceilings, at job-creation time rather than only at decode.
v0 — group H in
test/openjd/model_v0/v2023_09/test_feature_bundle_1.py:preprocess_job_parametersandcreate_job, parametrized over STRING, PATH, INT and FLOAT (the four types the merge re-validates)model_copybranch, so unifying the two branches cannot silently reintroduce the base limitIdentifierceiling rejects at 513, and driving the merge directly with no extensions still enforces 64 rather than skipping the checkv1 — a new
TestCreateJobPreservesFeatureBundle1Lengthsintest/openjd/model_v1/test_create_job.py. This lane had no coverage of these four ceilings at any level. Measured: v1 does not have the defect, for the structural reason above. These 22 tests are a ratchet, so a re-validation pass added upstream cannot land here unnoticed.Verification
hatch run test: 6108 passed, 24 skipped, 3 xfailed. 6070 before this change.hatch run lint,hatch run typing: clean.FEATURE_BUNDLE_1instead of deriving it, ignoring environment templates, applying the threading only to STRING, and applying it only to single-source merges.Not verified
The v1 tests cannot be mutation-checked from this repository — the implementation they exercise is the Rust extension, so the mutants for them would have to be applied in
openjd-rs. They are asserted to pass, not asserted to be falsifiable here.