MultiTab: A Comprehensive Benchmark Suite for Multi-Dimensional Evaluation in Tabular Domains
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916746232856576 |
|---|---|
| author | Lee, Kyungeun Eo, Moonjung Cho, Hye-Seung Kim, Dongmin Sim, Ye Seul Kim, Seoyoon Suh, Min-Kook Lim, Woohyung |
| author_facet | Lee, Kyungeun Eo, Moonjung Cho, Hye-Seung Kim, Dongmin Sim, Ye Seul Kim, Seoyoon Suh, Min-Kook Lim, Woohyung |
| contents | Despite the widespread use of tabular data in real-world applications, most benchmarks rely on average-case metrics, which fail to reveal how model behavior varies across diverse data regimes. To address this, we propose MultiTab, a benchmark suite and evaluation framework for multi-dimensional, data-aware analysis of tabular learning algorithms. Rather than comparing models only in aggregate, MultiTab categorizes 196 publicly available datasets along key data characteristics, including sample size, label imbalance, and feature interaction, and evaluates 13 representative models spanning a range of inductive biases. Our analysis shows that model performance is highly sensitive to such regimes: for example, models using sample-level similarity excel on datasets with large sample sizes or high inter-feature correlation, while models encoding inter-feature dependencies perform best with weakly correlated features. These findings reveal that inductive biases do not always behave as intended, and that regime-aware evaluation is essential for understanding and improving model behavior. MultiTab enables more principled model design and offers practical guidance for selecting models tailored to specific data characteristics. All datasets, code, and optimization logs are publicly available at https://huggingface.co/datasets/LGAI-DILab/Multitab. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_14312 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MultiTab: A Comprehensive Benchmark Suite for Multi-Dimensional Evaluation in Tabular Domains Lee, Kyungeun Eo, Moonjung Cho, Hye-Seung Kim, Dongmin Sim, Ye Seul Kim, Seoyoon Suh, Min-Kook Lim, Woohyung Machine Learning Artificial Intelligence Despite the widespread use of tabular data in real-world applications, most benchmarks rely on average-case metrics, which fail to reveal how model behavior varies across diverse data regimes. To address this, we propose MultiTab, a benchmark suite and evaluation framework for multi-dimensional, data-aware analysis of tabular learning algorithms. Rather than comparing models only in aggregate, MultiTab categorizes 196 publicly available datasets along key data characteristics, including sample size, label imbalance, and feature interaction, and evaluates 13 representative models spanning a range of inductive biases. Our analysis shows that model performance is highly sensitive to such regimes: for example, models using sample-level similarity excel on datasets with large sample sizes or high inter-feature correlation, while models encoding inter-feature dependencies perform best with weakly correlated features. These findings reveal that inductive biases do not always behave as intended, and that regime-aware evaluation is essential for understanding and improving model behavior. MultiTab enables more principled model design and offers practical guidance for selecting models tailored to specific data characteristics. All datasets, code, and optimization logs are publicly available at https://huggingface.co/datasets/LGAI-DILab/Multitab. |
| title | MultiTab: A Comprehensive Benchmark Suite for Multi-Dimensional Evaluation in Tabular Domains |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2505.14312 |