ReCycle: Fast and Efficient Long Time Series Forecasting with Residual Cyclic Transformers
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917658299990016 |
|---|---|
| author | Weyrauch, Arvid Steens, Thomas Taubert, Oskar Hanke, Benedikt Eqbal, Aslan Götz, Ewa Streit, Achim Götz, Markus Debus, Charlotte |
| author_facet | Weyrauch, Arvid Steens, Thomas Taubert, Oskar Hanke, Benedikt Eqbal, Aslan Götz, Ewa Streit, Achim Götz, Markus Debus, Charlotte |
| contents | Transformers have recently gained prominence in long time series forecasting by elevating accuracies in a variety of use cases. Regrettably, in the race for better predictive performance the overhead of model architectures has grown onerous, leading to models with computational demand infeasible for most practical applications. To bridge the gap between high method complexity and realistic computational resources, we introduce the Residual Cyclic Transformer, ReCycle. ReCycle utilizes primary cycle compression to address the computational complexity of the attention mechanism in long time series. By learning residuals from refined smoothing average techniques, ReCycle surpasses state-of-the-art accuracy in a variety of application use cases. The reliable and explainable fallback behavior ensured by simple, yet robust, smoothing average techniques additionally lowers the barrier for user acceptance. At the same time, our approach reduces the run time and energy consumption by more than an order of magnitude, making both training and inference feasible on low-performance, low-power and edge computing devices. Code is available at https://github.com/Helmholtz-AI-Energy/ReCycle |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_03429 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ReCycle: Fast and Efficient Long Time Series Forecasting with Residual Cyclic Transformers Weyrauch, Arvid Steens, Thomas Taubert, Oskar Hanke, Benedikt Eqbal, Aslan Götz, Ewa Streit, Achim Götz, Markus Debus, Charlotte Machine Learning Artificial Intelligence Transformers have recently gained prominence in long time series forecasting by elevating accuracies in a variety of use cases. Regrettably, in the race for better predictive performance the overhead of model architectures has grown onerous, leading to models with computational demand infeasible for most practical applications. To bridge the gap between high method complexity and realistic computational resources, we introduce the Residual Cyclic Transformer, ReCycle. ReCycle utilizes primary cycle compression to address the computational complexity of the attention mechanism in long time series. By learning residuals from refined smoothing average techniques, ReCycle surpasses state-of-the-art accuracy in a variety of application use cases. The reliable and explainable fallback behavior ensured by simple, yet robust, smoothing average techniques additionally lowers the barrier for user acceptance. At the same time, our approach reduces the run time and energy consumption by more than an order of magnitude, making both training and inference feasible on low-performance, low-power and edge computing devices. Code is available at https://github.com/Helmholtz-AI-Energy/ReCycle |
| title | ReCycle: Fast and Efficient Long Time Series Forecasting with Residual Cyclic Transformers |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2405.03429 |