user provided bound for torchtrt compile when export dimension is unb… by apbose · Pull Request #4213 · pytorch/TensorRT

apbose · 2026-04-27T19:27:22Z

This PR validates user bounds against any finite exporter bounds, with three branches:

Upper overflow (user_max > exp_max) → ValueError. TRT engine profile follows the exporter; shapes above exp_max would be rejected at runtime anyway.
Lower underflow (user_min < exp_min) → ValueError, except the (user_min=1, exp_min=2) case which is just PyTorch's 0/1-specialization artifact (Dim(min=1) silently becomes 2 in ShapeEnv), this particular case emits a warning.
Subset (user range strictly inside exporter range) → engine profile is narrowed to the user's bounds (logged at INFO). The user's Input(min_shape, max_shape) is the contract for the TRT profile; a subset of the exporter's valid range is still correct, and a tighter profile lets TRT specialize more aggressively.
Dim.DYNAMIC (unbounded exporter upper) → no check, user fills the gap (the intended use).

apbose · 2026-04-30T00:12:20Z

CI index put cases failing. Looking at it.

narendasan · 2026-05-06T21:18:00Z

This PR is only for externally exported programs?

apbose · 2026-05-06T21:26:25Z

@narendasan yeah it addresses (torch_tensorrt.dynamo.compile(exported_program, ...)). The flow for torch_tensorrt.compile(model, inputs=..) should be unaffected, since in that case export bounds and input bounds should be same

narendasan

You should use the user bounds in the narrow case

narendasan · 2026-06-02T23:08:14Z

+
+            if user_max > exp_max:
+                raise ValueError(
+                    f"{mismatch} Input.max_shape exceeds the exporter's max "


the exported program right?

yeah the exported program. will change the wording

narendasan · 2026-06-02T23:08:29Z

+            if user_max > exp_max:
+                raise ValueError(
+                    f"{mismatch} Input.max_shape exceeds the exporter's max "
+                    f"({user_max} > {exp_max}); TRT will reject shapes above "


TRT will reject or pytorch will?

TRT will. [10,20] + Input min_shape=2: shows
IExecutionContext::setInputShape: Error Code 3: API Usage Error
(Set dimension [2,8] for tensor x does not satisfy any optimization profiles.
Valid range for profile 0: [10,8]..[20,8])
through core/runtime/execute_engine.cpp:149.

narendasan · 2026-06-02T23:09:01Z

+                # 1->2 is the 0/1 specialization artifact, not a user error.
+                if user_min == 1 and exp_min == 2:
+                    logger.warning(
+                        "%s Input.min_shape=1 vs exporter min=2 is the "


I think we need to make sure these error messages are clear to follow from a user perspective

…ounded

…ympy.oo in extract_var_range_info, validate Input against finite export bounds

…specialization artifact, unify bound-mismatch message templates

…lders; sync assertions with refactored bound-mismatch messages

…orter bound

apbose · 2026-06-14T18:19:51Z

failing tests-
L0 dynamo core tests- RTX Linux- runtime cache failure
L0 dynamo core tests- Linux- same
L0 dynamo converter tests- RTX Linux- cumsum operator
L0 torchscript tests- windows- test_e2e_behavior.py worker crashes, test_get_layer_info Key Bindings missing
Python only runtime tests- wrong bounda

narendasan · 2026-06-15T20:38:23Z


+    # Forwarded to the partitioner to fill Dim.DYNAMIC upper bounds.
+    # Read-only w.r.t. ShapeEnv so range_constraints survive save/re-export.
+    user_symbol_bounds = _build_user_symbol_bounds(


instead of trying to handle args, kwargs, why not operate on the flattened pytree inputs? If you really do need to do this work this early then you can always call in_spec to flatten the args, kwargs. Then the order is deterministic and you arent doing name matching

narendasan · 2026-06-15T20:38:50Z

+    """
+    placeholders = [n for n in gm.graph.nodes if n.op == "placeholder"]
+
+    sample_by_name: dict[str, Input] = {}


Like this code should go away with pytree flattening and you can just zip the placeholders

narendasan · 2026-06-16T05:00:02Z

+
+    # Flatten args+kwargs in pytree order — guaranteed to match placeholder
+    # order by torch.export, so we can zip directly without name matching.
+    flat_inputs, _ = pytree.tree_flatten((list(sample_arg_inputs), sample_kwarg_inputs))


This should come from the exported program (i believe its called in_spec)

narendasan · 2026-06-16T05:00:47Z

+    log only); the ``user_min=1, exp_min=2`` case warns -- it's PyTorch's
+    0/1 specialization artifact, not a user error.
+    """
+    placeholders = [n for n in gm.graph.nodes if n.op == "placeholder"]


Same thing here, there should be a flat list of inputs to the graph

we would need the placeholders for the faketensor values right?

meta-cla Bot added the cla signed label Apr 27, 2026

github-actions Bot added component: tests Issues re: Tests component: core Issues re: The core compiler component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths labels Apr 27, 2026

github-actions Bot requested a review from lanluo-nvidia April 27, 2026 19:27

apbose force-pushed the abose/torchTRT_dynamic_user_shape_bounds branch from 4d1ff02 to eec2c04 Compare April 27, 2026 23:35

apbose force-pushed the abose/torchTRT_dynamic_user_shape_bounds branch from 0fe114e to 2905ff6 Compare April 30, 2026 01:05

apbose requested a review from narendasan May 6, 2026 21:09

apbose force-pushed the abose/torchTRT_dynamic_user_shape_bounds branch from dee44b1 to 22728b3 Compare May 6, 2026 22:15

narendasan reviewed May 7, 2026

View reviewed changes

apbose force-pushed the abose/torchTRT_dynamic_user_shape_bounds branch from 02d94e5 to 01798b0 Compare May 21, 2026 18:08

apbose force-pushed the abose/torchTRT_dynamic_user_shape_bounds branch from 01798b0 to fd52275 Compare May 28, 2026 20:56

narendasan reviewed Jun 2, 2026

View reviewed changes

apbose added 8 commits June 12, 2026 11:29

user provided bound for torchtrt compile when export dimension is unb…

afc27a7

…ounded

user passing input, adding test case

143ba02

fix(dynamo): honor Input(max_shape) for Dim.DYNAMIC dims, crash-fix s…

222f35c

…ympy.oo in extract_var_range_info, validate Input against finite export bounds

fix(dynamo): downgrade user_min<exp_min to warning, treat 1→2 as 0/1-…

93a1460

…specialization artifact, unify bound-mismatch message templates

use _first_sym_placeholder helper to skip lifted param/buffer placeho…

4047762

…lders; sync assertions with refactored bound-mismatch messages

cleaning up the comments

dc669ac

changing the narrower bound in case of user bound being subset of exp…

7544d23

…orter bound

addressed review comments making the messages clearer

69a0857

apbose force-pushed the abose/torchTRT_dynamic_user_shape_bounds branch from dae27b3 to 69a0857 Compare June 12, 2026 18:29

narendasan mentioned this pull request Jun 15, 2026

fix 4326: bound exported dynamic shapes and normalize symbolic slice … #4341

Open

7 tasks

narendasan reviewed Jun 15, 2026

View reviewed changes

Comment thread py/torch_tensorrt/dynamo/_compiler.py Outdated

narendasan reviewed Jun 15, 2026

View reviewed changes

Comment thread py/torch_tensorrt/dynamo/_compiler.py

addressing review comments

3c31af5

narendasan reviewed Jun 16, 2026

View reviewed changes

Conversation

apbose commented Apr 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

apbose commented Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

narendasan commented May 6, 2026

Uh oh!

apbose commented May 6, 2026

Uh oh!

narendasan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

apbose commented Jun 14, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

apbose commented Apr 27, 2026 •

edited

Loading

apbose commented Apr 30, 2026 •

edited

Loading