Ensembling Tabular Foundation Models - A Diversity Ceiling And A Calibration Trap
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910233000935424 |
|---|---|
| author | Tanna, Aditya Desai, Yash Seth, Pratinav Bouadi, Mohamed Bouarour, Nassim Sankarapu, Vinay Kumar |
| author_facet | Tanna, Aditya Desai, Yash Seth, Pratinav Bouadi, Mohamed Bouarour, Nassim Sankarapu, Vinay Kumar |
| contents | Tabular foundation models (TFMs) now match or beat tuned gradient-boosted trees on a growing fraction of tabular tasks, but no single TFM wins on every dataset. Ensembling is the go to fix here, and it works less well than expected. Six modern TFMs form a near-redundant pool: their mean pairwise Q-statistic is $0.961$, close enough to $1$ that any convex combination is bounded above. We benchmark six ensemble strategies over six TFMs on 153 OpenML classification tasks. The best ensemble, two-level cascade stacking, buys $+0.18\%$ accuracy over the strongest single TFM at $253\times$ the compute. A Friedman and Nemenyi analysis places three ensembles and the best base TFM in a single equivalence group; three other ensembles are significantly \emph{worse} than the best base. Stacking with a logistic-regression meta-learner is the most striking case: competitive accuracy and ROC-AUC, the worst log-loss rank among the ensembles. The meta-learner improves accuracy by sharpening class boundaries, which destroys calibration. We recommend greedy selection as the practical default. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_18696 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Ensembling Tabular Foundation Models - A Diversity Ceiling And A Calibration Trap Tanna, Aditya Desai, Yash Seth, Pratinav Bouadi, Mohamed Bouarour, Nassim Sankarapu, Vinay Kumar Machine Learning Artificial Intelligence Tabular foundation models (TFMs) now match or beat tuned gradient-boosted trees on a growing fraction of tabular tasks, but no single TFM wins on every dataset. Ensembling is the go to fix here, and it works less well than expected. Six modern TFMs form a near-redundant pool: their mean pairwise Q-statistic is $0.961$, close enough to $1$ that any convex combination is bounded above. We benchmark six ensemble strategies over six TFMs on 153 OpenML classification tasks. The best ensemble, two-level cascade stacking, buys $+0.18\%$ accuracy over the strongest single TFM at $253\times$ the compute. A Friedman and Nemenyi analysis places three ensembles and the best base TFM in a single equivalence group; three other ensembles are significantly \emph{worse} than the best base. Stacking with a logistic-regression meta-learner is the most striking case: competitive accuracy and ROC-AUC, the worst log-loss rank among the ensembles. The meta-learner improves accuracy by sharpening class boundaries, which destroys calibration. We recommend greedy selection as the practical default. |
| title | Ensembling Tabular Foundation Models - A Diversity Ceiling And A Calibration Trap |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2605.18696 |