Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2410.13516 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909353421832192 |
|---|---|
| author | Spinaci, Marco Polewczyk, Marek Hoffart, Johannes Kohler, Markus C. Thelin, Sam Klein, Tassilo |
| author_facet | Spinaci, Marco Polewczyk, Marek Hoffart, Johannes Kohler, Markus C. Thelin, Sam Klein, Tassilo |
| contents | Self-supervised learning on tabular data seeks to apply advances from natural language and image domains to the diverse domain of tables. However, current techniques often struggle with integrating multi-domain data and require data cleaning or specific structural requirements, limiting the scalability of pre-training datasets. We introduce PORTAL (Pretraining One-Row-at-a-Time for All tabLes), a framework that handles various data modalities without the need for cleaning or preprocessing. This simple yet powerful approach can be effectively pre-trained on online-collected datasets and fine-tuned to match state-of-the-art methods on complex classification and regression tasks. This work offers a practical advancement in self-supervised learning for large-scale tabular data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_13516 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PORTAL: Scalable Tabular Foundation Models via Content-Specific Tokenization Spinaci, Marco Polewczyk, Marek Hoffart, Johannes Kohler, Markus C. Thelin, Sam Klein, Tassilo Machine Learning Self-supervised learning on tabular data seeks to apply advances from natural language and image domains to the diverse domain of tables. However, current techniques often struggle with integrating multi-domain data and require data cleaning or specific structural requirements, limiting the scalability of pre-training datasets. We introduce PORTAL (Pretraining One-Row-at-a-Time for All tabLes), a framework that handles various data modalities without the need for cleaning or preprocessing. This simple yet powerful approach can be effectively pre-trained on online-collected datasets and fine-tuned to match state-of-the-art methods on complex classification and regression tasks. This work offers a practical advancement in self-supervised learning for large-scale tabular data. |
| title | PORTAL: Scalable Tabular Foundation Models via Content-Specific Tokenization |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2410.13516 |