Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914111549341696 |
|---|---|
| author | Zhang, Xiyuan Maddix, Danielle C. Yin, Junming Erickson, Nick Ansari, Abdul Fatir Han, Boran Zhang, Shuai Akoglu, Leman Faloutsos, Christos Mahoney, Michael W. Hu, Cuixiong Rangwala, Huzefa Karypis, George Wang, Bernie |
| author_facet | Zhang, Xiyuan Maddix, Danielle C. Yin, Junming Erickson, Nick Ansari, Abdul Fatir Han, Boran Zhang, Shuai Akoglu, Leman Faloutsos, Christos Mahoney, Michael W. Hu, Cuixiong Rangwala, Huzefa Karypis, George Wang, Bernie |
| contents | Since the seminal work of TabPFN, research on tabular foundation models (TFMs) based on in-context learning (ICL) has challenged long-standing paradigms in machine learning. Without seeing any real-world data, models pretrained on purely synthetic datasets generalize remarkably well across diverse datasets, often using only a moderate number of in-context examples. This shifts the focus in tabular machine learning from model architecture design to the design of synthetic datasets, or, more precisely, to the prior distributions that generate them. Yet the guiding principles for prior design remain poorly understood. This work marks the first attempt to address the gap. We systematically investigate and identify key properties of synthetic priors that allow pretrained TFMs to generalize well. Based on these insights, we introduce Mitra, a TFM trained on a curated mixture of synthetic priors selected for their diversity, distinctiveness, and performance on real-world tabular data. Mitra consistently outperforms state-of-the-art TFMs, such as TabPFNv2 and TabICL, across both classification and regression benchmarks, with better sample efficiency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_21204 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Models Zhang, Xiyuan Maddix, Danielle C. Yin, Junming Erickson, Nick Ansari, Abdul Fatir Han, Boran Zhang, Shuai Akoglu, Leman Faloutsos, Christos Mahoney, Michael W. Hu, Cuixiong Rangwala, Huzefa Karypis, George Wang, Bernie Machine Learning Since the seminal work of TabPFN, research on tabular foundation models (TFMs) based on in-context learning (ICL) has challenged long-standing paradigms in machine learning. Without seeing any real-world data, models pretrained on purely synthetic datasets generalize remarkably well across diverse datasets, often using only a moderate number of in-context examples. This shifts the focus in tabular machine learning from model architecture design to the design of synthetic datasets, or, more precisely, to the prior distributions that generate them. Yet the guiding principles for prior design remain poorly understood. This work marks the first attempt to address the gap. We systematically investigate and identify key properties of synthetic priors that allow pretrained TFMs to generalize well. Based on these insights, we introduce Mitra, a TFM trained on a curated mixture of synthetic priors selected for their diversity, distinctiveness, and performance on real-world tabular data. Mitra consistently outperforms state-of-the-art TFMs, such as TabPFNv2 and TabICL, across both classification and regression benchmarks, with better sample efficiency. |
| title | Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Models |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2510.21204 |