OmniPred: Language Models as Universal Regressors
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912211971080192 |
|---|---|
| author | Song, Xingyou Li, Oscar Lee, Chansoo Yang, Bangding Peng, Daiyi Perel, Sagi Chen, Yutian |
| author_facet | Song, Xingyou Li, Oscar Lee, Chansoo Yang, Bangding Peng, Daiyi Perel, Sagi Chen, Yutian |
| contents | Regression is a powerful tool to accurately predict the outcome metric of a system given a set of parameters, but has traditionally been restricted to methods which are only applicable to a specific task. In this paper, we propose OmniPred, a framework for training language models as universal end-to-end regressors over $(x,y)$ data from arbitrary formats. Using data sourced from Google Vizier, one of the largest proprietary blackbox optimization databases in the world, our extensive experiments demonstrate that language models are capable of very precise numerical regression using only textual representations of mathematical parameters and values, and if given the opportunity to train at scale over multiple tasks, can significantly outperform traditional regression models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_14547 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | OmniPred: Language Models as Universal Regressors Song, Xingyou Li, Oscar Lee, Chansoo Yang, Bangding Peng, Daiyi Perel, Sagi Chen, Yutian Machine Learning Artificial Intelligence Computation and Language Databases Regression is a powerful tool to accurately predict the outcome metric of a system given a set of parameters, but has traditionally been restricted to methods which are only applicable to a specific task. In this paper, we propose OmniPred, a framework for training language models as universal end-to-end regressors over $(x,y)$ data from arbitrary formats. Using data sourced from Google Vizier, one of the largest proprietary blackbox optimization databases in the world, our extensive experiments demonstrate that language models are capable of very precise numerical regression using only textual representations of mathematical parameters and values, and if given the opportunity to train at scale over multiple tasks, can significantly outperform traditional regression models. |
| title | OmniPred: Language Models as Universal Regressors |
| topic | Machine Learning Artificial Intelligence Computation and Language Databases |
| url | https://arxiv.org/abs/2402.14547 |