From 442412c5ee99d349f00ce5ccb58305e7c403dca9 Mon Sep 17 00:00:00 2001 From: Andy Grove Date: Mon, 3 Aug 2026 15:52:13 -0600 Subject: [PATCH] docs: document ReusedExchange caveat and both CometSparkToColumnar names in operator-count exclusions Follow-up to review feedback on #5206. - The transition-nodes bullet only listed `CometSparkRowToColumnar`, but `CometSparkToColumnarExec` also renders as `CometSparkColumnarToColumnar` when its child supports columnar. Both names are excluded from the operator counts today (the match is on the Scala type, not the rendered name), so list both. - `ReusedExchangeExec` is matched in the same ignore arm as the other wrapper nodes but was not mentioned. It also behaves differently from them: `getActualPlan` unwraps it to its child, so a reused subtree is counted once per reference rather than once per plan. Document it along with that caveat and the link to item 3 of #5203. --- .../user-guide/latest/understanding-comet-plans.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/docs/source/user-guide/latest/understanding-comet-plans.md b/docs/source/user-guide/latest/understanding-comet-plans.md index a05d271360..ea32e37b8f 100644 --- a/docs/source/user-guide/latest/understanding-comet-plans.md +++ b/docs/source/user-guide/latest/understanding-comet-plans.md @@ -182,12 +182,22 @@ Not every node in the plan is an eligible operator. The following are excluded from both operator counts: - Transition nodes (`CometColumnarToRow`, `CometNativeColumnarToRow`, - `CometSparkRowToColumnar`, `ColumnarToRow`, `RowToColumnar`), which are - reported separately as the transition count. + `CometSparkRowToColumnar`, `CometSparkColumnarToColumnar`, `ColumnarToRow`, + `RowToColumnar`), which are reported separately as the transition count. + `CometSparkRowToColumnar` and `CometSparkColumnarToColumnar` are the two names + a single operator renders under, depending on whether its child already + produces columnar data, and both are excluded. - Wrappers that do no work of their own: `AdaptiveSparkPlan`, `InputAdapter`, `WholeStageCodegen`, query stages, and `AQEShuffleRead`. - The reuse marker `ReusedSubquery`. The subquery it points at is counted where that subquery is shown, so the marker itself does not add to the totals. +- `ReusedExchange`, but with a caveat: it is not cleanly excluded the way the + wrappers above are. The node itself is skipped, and yet walking the plan + replaces it with the exchange it reuses, so the reused subtree is counted once + per reference rather than once for the whole plan. A plan that reuses one + exchange in three places contributes that subtree's operators three times. + Counting reused exchanges once is tracked as item 3 of + [#5203](https://github.com/apache/datafusion-comet/issues/5203). ### `spark.comet.explain.native.enabled`