Conversation
Users pass connection ids through Params and Variables to reuse one Dag across environments, which only works when the operator lists the argument in template_fields. Coverage was uneven: a third of provider operators templated their connection id, the rest did not, and subclasses that redefined template_fields silently dropped the parent's connection id. Templating is added wherever it can take effect: arguments that are read in __init__ or stored under a different attribute name are left alone.
Without a check the previous commit erodes: new operators forget the argument, and subclasses that redefine template_fields drop the parent's connection id. The hook resolves template_fields across the inheritance chain the same way Python does, and skips arguments templating cannot act on (read in __init__, or never stored under the argument name).
shahar1
marked this pull request as ready for review
September 17, 2026 13:08
shahar1
requested review from
ashb,
bolkedebruin,
gopidesupavan and
potiuk
as code owners
September 17, 2026 13:08
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes: #35259
Human Summary
I've figured out that it would be easier and better to add all
conn_idat once as template fields to all operators, as the original issue suggests, rather than managing a "burn-out" operation like we did in #70296. It is a very quick fix, and I don't want to spend days reviewing AI PRs that do it over and over again. To ensure that it is being enforced, also added a pre-commit for that.AI Summary
Click here
Connection ids are meant to be templatable (`conn_id="{{ params.conn_id }}"`), but coverage across providers was uneven: 331 of 1144 provider classes with `template_fields` listed their `*conn_id` argument, the rest did not, and many subclasses that redefine `template_fields` silently dropped the parent's connection id.Commit 1 adds every
*conn_idargument totemplate_fieldswhere templating can take effect (523 entries across 237 files, 59 providers). Arguments read in__init__or stored under a different attribute name (e.g.SparkSubmitOperatorkeeps_conn_id) are left alone since rendering would not reach them. Six Vertex AI operators gain an explicitgcp_conn_idparameter forwarded to the base class sovalidate-operators-initsees a plain assignment. Five tests asserting exact tuples are updated.Commit 2 adds the
check-conn-id-templatedprek hook (providers config) so this does not erode. It parses the provider tree once, resolvestemplate_fieldsacross the inheritance chain the way Python does (literal tuples,*Parent.template_fields,tuple({...} | set(Parent.template_fields)),aws_template_fields(...)), and reports any*conn_id__init__argument, own or inherited, that the resolved fields miss. Arguments templating cannot act on are skipped. Runs in ~6 s on the full tree, only parses files containingconn_id. Covered by 16 unit tests underscripts/tests/ci/prek/.Verified with
breeze run mypyon the touched files, the runtime superset check of every edited class, and thetemplate_fieldstests of all 59 providers.related: #35259
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Fable 5.1) following the guidelines
🤖 Generated with Claude Code