Analysing Multi-Task Regression via Random Matrix Theory with Application to Time Series Forecasting
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913391815163904 |
|---|---|
| author | Ilbert, Romain Tiomoko, Malik Louart, Cosme Odonnat, Ambroise Feofanov, Vasilii Palpanas, Themis Redko, Ievgen |
| author_facet | Ilbert, Romain Tiomoko, Malik Louart, Cosme Odonnat, Ambroise Feofanov, Vasilii Palpanas, Themis Redko, Ievgen |
| contents | In this paper, we introduce a novel theoretical framework for multi-task regression, applying random matrix theory to provide precise performance estimations, under high-dimensional, non-Gaussian data distributions. We formulate a multi-task optimization problem as a regularization technique to enable single-task models to leverage multi-task learning information. We derive a closed-form solution for multi-task optimization in the context of linear models. Our analysis provides valuable insights by linking the multi-task learning performance to various model statistics such as raw data covariances, signal-generating hyperplanes, noise levels, as well as the size and number of datasets. We finally propose a consistent estimation of training and testing errors, thereby offering a robust foundation for hyperparameter optimization in multi-task regression scenarios. Experimental validations on both synthetic and real-world datasets in regression and multivariate time series forecasting demonstrate improvements on univariate models, incorporating our method into the training loss and thus leveraging multivariate information. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_10327 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Analysing Multi-Task Regression via Random Matrix Theory with Application to Time Series Forecasting Ilbert, Romain Tiomoko, Malik Louart, Cosme Odonnat, Ambroise Feofanov, Vasilii Palpanas, Themis Redko, Ievgen Machine Learning In this paper, we introduce a novel theoretical framework for multi-task regression, applying random matrix theory to provide precise performance estimations, under high-dimensional, non-Gaussian data distributions. We formulate a multi-task optimization problem as a regularization technique to enable single-task models to leverage multi-task learning information. We derive a closed-form solution for multi-task optimization in the context of linear models. Our analysis provides valuable insights by linking the multi-task learning performance to various model statistics such as raw data covariances, signal-generating hyperplanes, noise levels, as well as the size and number of datasets. We finally propose a consistent estimation of training and testing errors, thereby offering a robust foundation for hyperparameter optimization in multi-task regression scenarios. Experimental validations on both synthetic and real-world datasets in regression and multivariate time series forecasting demonstrate improvements on univariate models, incorporating our method into the training loss and thus leveraging multivariate information. |
| title | Analysing Multi-Task Regression via Random Matrix Theory with Application to Time Series Forecasting |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2406.10327 |