Skip to content

Propagate aggregate-result statistics into HAVING #25615

Description

@gabotechs

Describe the bug

The SUM result has no useful statistics for a downstream HAVING filter. A
simplified Q18 query separates this loss from the already inaccurate group-count
estimate.

To Reproduce

From the repository root, with the CLI fix from
PR #25570 applied:

cargo build --profile ci --locked -p datafusion-benchmarks --bin dfbench
cargo install tpchgen-cli --version 1.1.1 --locked # if not already installed
repro_dir=$(mktemp -d)
tpchgen-cli --scale-factor 1 --format parquet \
  --parquet-compression 'ZSTD(1)' --parts 1 --output-dir "$repro_dir/data"
cat > "$repro_dir/repro.sql" <<'SQL'
SET datafusion.execution.target_partitions = 1;
SET datafusion.optimizer.enable_dynamic_filter_pushdown = false;
SELECT l_orderkey, SUM(l_quantity) AS qty FROM lineitem GROUP BY l_orderkey HAVING SUM(l_quantity) > 300;
SQL
target/ci/dfbench statistics \
  --path "$repro_dir/data" --query_path "$repro_dir/repro.sql"

Observed with tpchgen-cli 1.1.1 at
6c320561b5.
Inspect the SELECT reports; ignore the empty SET reports.

Operator Node Estimated rows Actual rows
AggregateExec 0.0.0 6,001,215 1,500,000
FilterExec (SUM > 300) 0.0 1,200,243 57

Expected behavior

Propagate conservative aggregate-result statistics and use them for downstream
selectivity. Rare SUM tails may require distribution summaries; bounds alone are
insufficient.

Additional context

The q-error grows from 4× at the aggregate to 21,057× at the filter. Improving
group counts alone does not model the SUM distribution.

Part of #25610.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions