Low-rank attention augmented Gaussian processes for multivariate data analysis
Fuente:
Zenodo
Guardado en:
| Autor principal: | |
|---|---|
| Formato: | Recurso digital |
| Publicado: |
Zenodo
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866902247487569920 |
|---|---|
| author | Oyebamiji, Oluwole |
| author_facet | Oyebamiji, Oluwole |
| contents | <p><span>We have developed an efficient low-rank attention-augmented Gaussian processes (LAAGP) </span><span>model that effectively combines accuracy with a reduction in the computational costs associated with transformer attention and Gaussian processes (GP). This model addresses the limitations of standard GP models, such as poor covariance function expressiveness for long-range multivariate forecasting and inadequate </span><span>data representation capacity. LAAGP is a powerful forecasting technique that integrates the transformer </span><span>self-attention mechanism with GP. The framework features a transformer encoder that processes the input </span><span>embeddings to extract essential information, using positional and variable encoding along with relative </span><span>embeddings to enhance attention scores. The GP decoder, known for its flexibility and reliable uncertainty </span><span>estimates, has been adapted to predict the system’s evolution over time. This enhancement enables the model to achieve a balance between computational efficiency, predictive accuracy, and uncertainty quantification, thereby enhancing performance on complex tasks such as</span><span> long-range predictions. Our model has been evaluated </span><span>on several benchmark regression and classification datasets.</span></p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_17385441 |
| institution | Zenodo |
| language | |
| publishDate | 2025 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | Low-rank attention augmented Gaussian processes for multivariate data analysis Oyebamiji, Oluwole Low rank approximation Gaussian process Transformer <p><span>We have developed an efficient low-rank attention-augmented Gaussian processes (LAAGP) </span><span>model that effectively combines accuracy with a reduction in the computational costs associated with transformer attention and Gaussian processes (GP). This model addresses the limitations of standard GP models, such as poor covariance function expressiveness for long-range multivariate forecasting and inadequate </span><span>data representation capacity. LAAGP is a powerful forecasting technique that integrates the transformer </span><span>self-attention mechanism with GP. The framework features a transformer encoder that processes the input </span><span>embeddings to extract essential information, using positional and variable encoding along with relative </span><span>embeddings to enhance attention scores. The GP decoder, known for its flexibility and reliable uncertainty </span><span>estimates, has been adapted to predict the system’s evolution over time. This enhancement enables the model to achieve a balance between computational efficiency, predictive accuracy, and uncertainty quantification, thereby enhancing performance on complex tasks such as</span><span> long-range predictions. Our model has been evaluated </span><span>on several benchmark regression and classification datasets.</span></p> |
| title | Low-rank attention augmented Gaussian processes for multivariate data analysis |
| topic | Low rank approximation Gaussian process Transformer |
| url | https://doi.org/10.5281/zenodo.17385441 |