feat(cuda.core): support nvJitLink incremental linking by isVoid · Pull Request #2867 · NVIDIA/cuda-python

isVoid · 2026-09-15T18:55:10Z

Summary

expose nvJitLink incremental linking as LinkerOptions(incremental=True), gated on nvJitLink 13.2+
return native partial results as ObjectCode(code_type="cubin") so they round-trip directly into another Linker
add Linker.link("ltoir") on nvJitLink 13.3+ so incremental LTO chains can retain IR until final code generation
reject unsupported driver-backend and partial-PTX combinations with explicit errors
clarify that code_type="object" means a host object containing device code

Design choices

A native incremental-link result remains "cubin"; ELF ET_REL describes link state, not a distinct cuda.core transport format.
In incremental mode, target_type accepts "cubin", or "ltoir" when LTO is enabled and the getter is available. "ptx" is rejected.
Native CUBIN and linked-LTOIR outputs need no wrapper or conversion before being supplied to a later Linker.
Intermediate LTO stages should request "ltoir"; requesting "cubin" creates a machine-code boundary that later LTO cannot optimize through.
Linked-LTOIR retrieval uses the Python binding dynamically so cuda-core continues to build with supported cuda-bindings 12.x releases whose Cython declarations predate the getter.

Testing

CUDA 13.4 linker suite: 82 passed, 3 skipped, including executable native-CUBIN and LTOIR three-stage round trips
CUDA 12.9 / published cuda-bindings 12.9.7 compatibility build and linker suite: 74 passed, 6 capability skips
Sphinx documentation build
repository pre-commit checks for all changed files (Ruff, mypy, Cython lint, generated stubs, SPDX, RST, memory-pool hygiene)

Closes #2369

copy-pr-bot · 2026-09-15T18:55:14Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

leofang · 2026-09-15T19:35:21Z

nit: @isVoid could you post your design doc as a comment in #2369 for posterity, in case your personal branch is removed in the future?

isVoid · 2026-09-17T18:17:53Z

nit: @isVoid could you post your design doc as a comment in #2369 for posterity, in case your personal branch is removed in the future?

This is now posted.

leofang · 2026-09-17T20:38:03Z

    link_time_optimization : bool, optional
        Perform link time optimization.
        Default: False.
+    relocatable : bool, optional


Q: Maybe we should call it relocatable_device_code to match what we have in ProgramOptions? Seems like they have the same semantics?

The design doc notes the distinction:

ProgramOptions.relocatable_device_code and the proposed LinkerOptions.relocatable are also distinct:

The program option makes a compilation unit suitable for device linking.

The linker option permits a link result to remain relocatable and to retain unresolved device references.

I sort of think relocatable_device_code and relocatable are respectively different notions created one on the nvrtc and nvjitlink side. My prior discussion with folks suggests rdc is quite commonly used as a shorthand to refer to the nvrtc behavior. Coercing the linker side to overlap this terminology could introduce some confusion in the future.

ref:
nvjitlink linker options
nvrtc options

I think I was confused for a moment when reading even relocatable_device_code and relocatable 😂

@isVoid good idea, but maybe we should rename it for clarity? Such as incremental_link?

+1. Like incremental_link

Maybe the page didn't refresh when you reply, see #2867 (comment). Do you have specific preference on incremental_link over incremental? @lijinf2

Yes I think page did not refresh on my side. incremental looks good too!

Don't the driver docs prefer the term "relative" over "incremental" though? Diverging from that could be confusing in a different way.

Don't the driver docs prefer the term "relative" over "incremental" though? Diverging from that could be confusing in a different way.

I can see the term relocatable is used across the board to refer specifically to the output of the linker with -r or compiler with -rdc. But when referring to the -r flag, the nvjitlink do mention both terms, and actually in some sense highlighting the incremental to emphasize the linker behavior.

Andy-Jost

One blocker: link("ltoir") silently drops inputs that carry no LTOIR (details inline, with a repro). Two untested rows of the design matrix, and a few small cleanups. The relocatable and LTOIR round trips themselves look right.

Andy-Jost · 2026-09-17T22:36:03Z

    link_time_optimization : bool, optional
        Perform link time optimization.
        Default: False.
+    relocatable : bool, optional


The design doc notes the distinction:

ProgramOptions.relocatable_device_code and the proposed LinkerOptions.relocatable are also distinct:

The program option makes a compilation unit suitable for device linking.

The linker option permits a link result to remain relocatable and to retain unresolved device references.

lijinf2

Nice. Let me write down my understanding. Before this PR, application code had to gather all compiled device inputs (PTX/LTOIR/cubin) before invoking a single Linker. After this PR, application code can do cascading linking — each stage produces a partial result that feeds directly into the next Linker.

lijinf2 · 2026-09-18T01:24:28Z

    link_time_optimization : bool, optional
        Perform link time optimization.
        Default: False.
+    relocatable : bool, optional


I think I was confused for a moment when reading even relocatable_device_code and relocatable 😂

Andy-Jost

LGTM

leofang · 2026-09-18T17:42:28Z

Left a question here: #2867 (comment)

isVoid · 2026-09-18T18:49:24Z

/ok to test 3cc270d

lijinf2

My comments all addressed. Thanks. PR looks good to me!

isVoid · 2026-09-19T00:41:30Z

/ok to test fc8c850

github-actions · 2026-09-19T02:48:02Z

Doc Preview CI
Preview removed because the pull request was closed or merged.

feat(cuda.core): support incremental linking

4b04d0c

github-actions Bot added the cuda.core Everything related to the cuda.core module label Sep 15, 2026

leofang assigned isVoid Sep 15, 2026

leofang self-requested a review September 15, 2026 19:36

isVoid marked this pull request as ready for review September 17, 2026 20:09

leofang reviewed Sep 17, 2026

View reviewed changes

Andy-Jost requested changes Sep 17, 2026

View reviewed changes

Michael Wang added 3 commits September 18, 2026 00:21

fix(cuda.core): reject lossy LTOIR output

dc36fd7

test(cuda.core): cover incremental LTO output modes

129f1dd

refactor(cuda.core): remove redundant LTOIR error handling

073ba15

lijinf2 reviewed Sep 18, 2026

View reviewed changes

Michael Wang added 3 commits September 18, 2026 03:35

refactor(cuda.core): cache nvJitLink runtime details

068022e

chore(cuda.core): track linked LTOIR cimport cleanup

dd6761b

test(cuda.core): execute resolved relocatable cubin

3a2b1e4

isVoid added the feature New feature or request label Sep 18, 2026

isVoid added this to the cuda.core 1.3.0 milestone Sep 18, 2026

isVoid added the P1 Medium priority - Should do label Sep 18, 2026

isVoid requested a review from Andy-Jost September 18, 2026 06:02

Andy-Jost approved these changes Sep 18, 2026

View reviewed changes

leofang requested changes Sep 18, 2026

View reviewed changes

Comment thread cuda_core/cuda/core/LINKER_INCREMENTAL_LINKING_DESIGN.md Outdated

isVoid added 2 commits September 18, 2026 17:43

docs(cuda.core): remove incremental linking design document

8d9e505

refactor(cuda.core): rename incremental linker option

3cc270d

lijinf2 approved these changes Sep 18, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

test(cuda.core): align nvJitLink version cache coverage

fc8c850

Andy-Jost mentioned this pull request Sep 18, 2026

cuda.core: require a cuda-bindings floor and call the driver through its resolved pointers (#2783) #2920

Draft

isVoid enabled auto-merge (squash) September 19, 2026 00:46

isVoid disabled auto-merge September 19, 2026 00:46

isVoid requested a review from leofang September 19, 2026 00:46

leofang approved these changes Sep 19, 2026

View reviewed changes

leofang merged commit 8b6e9f5 into NVIDIA:main Sep 19, 2026
115 checks passed

Conversation

isVoid commented Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Design choices

Testing

Uh oh!

copy-pr-bot Bot commented Sep 15, 2026

Uh oh!

leofang commented Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

isVoid commented Sep 17, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

isVoid Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Andy-Jost left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lijinf2 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Andy-Jost left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

leofang commented Sep 18, 2026

Uh oh!

isVoid commented Sep 18, 2026

Uh oh!

lijinf2 left a comment

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

isVoid commented Sep 19, 2026

Uh oh!

Uh oh!

github-actions Bot commented Sep 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

isVoid commented Sep 15, 2026 •

edited

Loading

leofang commented Sep 15, 2026 •

edited

Loading

isVoid Sep 18, 2026 •

edited

Loading

lijinf2 left a comment •

edited

Loading