Skip to content

docs: document ReusedExchange caveat and both CometSparkToColumnar names in operator-count exclusions #5240

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
andygrove merged 1 commit into from
Aug 3, 2026
+12 −2
Merged

docs: document ReusedExchange caveat and both CometSparkToColumnar names in operator-count exclusions #5240

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions docs/source/user-guide/latest/understanding-comet-plans.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -182,12 +182,22 @@ Not every node in the plan is an eligible operator. The following are excluded
from both operator counts:

- Transition nodes (`CometColumnarToRow`, `CometNativeColumnarToRow`,
`CometSparkRowToColumnar`, `ColumnarToRow`, `RowToColumnar`), which are
reported separately as the transition count.
`CometSparkRowToColumnar`, `CometSparkColumnarToColumnar`, `ColumnarToRow`,
`RowToColumnar`), which are reported separately as the transition count.
`CometSparkRowToColumnar` and `CometSparkColumnarToColumnar` are the two names
a single operator renders under, depending on whether its child already
produces columnar data, and both are excluded.
- Wrappers that do no work of their own: `AdaptiveSparkPlan`, `InputAdapter`,
`WholeStageCodegen`, query stages, and `AQEShuffleRead`.
- The reuse marker `ReusedSubquery`. The subquery it points at is counted where
that subquery is shown, so the marker itself does not add to the totals.
- `ReusedExchange`, but with a caveat: it is not cleanly excluded the way the
wrappers above are. The node itself is skipped, and yet walking the plan
replaces it with the exchange it reuses, so the reused subtree is counted once
per reference rather than once for the whole plan. A plan that reuses one
exchange in three places contributes that subtree's operators three times.
Counting reused exchanges once is tracked as item 3 of
[#5203](https://github.com/apache/datafusion-comet/issues/5203).

### `spark.comet.explain.native.enabled`

Expand Down