Skip to content

Use 8 estimators for small and medium KumoTabular in benchmarks - #991

Merged
JingangQu merged 1 commit into
mainfrom
kumo-small-medium-estimators
Sep 26, 2026
Merged

JingangQu merged 1 commit into
mainfrom
kumo-small-medium-estimators

Conversation

@JingangQu

Copy link
Copy Markdown
Collaborator

#983 raised every size to 16 estimators. This PR corrects it and sets 8 estimators by default for small and medium KumoTabular in the TabArena/BeyondArena, ScoringBench and TALENT benchmarks. Large keeps 16.

The size options from #983 set 16 estimators for every size; large
keeps 16.

Signed-off-by: Jingang Qu <jqu@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Sep 26, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: NVIDIA/structured-data-models/.coderabbit.yaml

Review profile: QUIET

Plan: Enterprise

Run ID: 8a783334-99e6-4f17-9c17-d61eedf627dc

📥 Commits

Reviewing files that changed from the base of the PR and between f8cdb27 and bad1e66.

📒 Files selected for processing (3)
  • benchmark/tabular/model.py
  • benchmark/tabular/scoringbench/models.py
  • benchmark/tabular/talent/models.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.


📝 Summary

Summary by CodeRabbit

  • Updates
    • Small and medium Kumo tabular benchmark configurations now use 8 estimators instead of 16.
    • Large Kumo tabular benchmark configurations remain at 16 estimators.

Walkthrough

The medium model default and small and medium benchmark configurations now use 8 estimators instead of 16. The large configuration and other described settings are unchanged.

Changes

KumoTabular estimator count

Layer / File(s) Summary
Set estimator counts
benchmark/tabular/model.py, benchmark/tabular/scoringbench/models.py, benchmark/tabular/talent/models.py
The medium model default and small and medium benchmark configurations now use 8 estimators. The large configuration and other described settings are unchanged.

Priority: ⬇️ Low

Estimated code review effort: 1 (Trivial) | ~4 minutes

Merge Risk: ⚪ Minimal · up to bad1e

The estimator defaults and benchmark configurations match the stated size-specific counts. No merge-blocking issue was identified.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 3…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly states the main change: using 8 estimators for small and medium KumoTabular benchmark configurations.
Description check ✅ Passed The description accurately explains the estimator change, affected benchmarks, and that large KumoTabular remains at 16 estimators.
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@JingangQu
JingangQu merged commit 5d2dfb6 into main Sep 26, 2026
4 checks passed
@JingangQu
JingangQu deleted the kumo-small-medium-estimators branch September 26, 2026 09:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant