Less is more: Embracing sparsity and interpolation with Esiformer for time series forecasting
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912064035880960 |
|---|---|
| author | Guo, Yangyang Zhao, Yanjun Dang, Sizhe Zhou, Tian Sun, Liang Qian, Yi |
| author_facet | Guo, Yangyang Zhao, Yanjun Dang, Sizhe Zhou, Tian Sun, Liang Qian, Yi |
| contents | Time series forecasting has played a significant role in many practical fields. But time series data generated from real-world applications always exhibits high variance and lots of noise, which makes it difficult to capture the inherent periodic patterns of the data, hurting the prediction accuracy significantly. To address this issue, we propose the Esiformer, which apply interpolation on the original data, decreasing the overall variance of the data and alleviating the influence of noise. What's more, we enhanced the vanilla transformer with a robust Sparse FFN. It can enhance the representation ability of the model effectively, and maintain the excellent robustness, avoiding the risk of overfitting compared with the vanilla implementation. Through evaluations on challenging real-world datasets, our method outperforms leading model PatchTST, reducing MSE by 6.5% and MAE by 5.8% in multivariate time series forecasting. Code is available at: https://github.com/yyg1282142265/Esiformer/tree/main. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_05726 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Less is more: Embracing sparsity and interpolation with Esiformer for time series forecasting Guo, Yangyang Zhao, Yanjun Dang, Sizhe Zhou, Tian Sun, Liang Qian, Yi Machine Learning Artificial Intelligence Time series forecasting has played a significant role in many practical fields. But time series data generated from real-world applications always exhibits high variance and lots of noise, which makes it difficult to capture the inherent periodic patterns of the data, hurting the prediction accuracy significantly. To address this issue, we propose the Esiformer, which apply interpolation on the original data, decreasing the overall variance of the data and alleviating the influence of noise. What's more, we enhanced the vanilla transformer with a robust Sparse FFN. It can enhance the representation ability of the model effectively, and maintain the excellent robustness, avoiding the risk of overfitting compared with the vanilla implementation. Through evaluations on challenging real-world datasets, our method outperforms leading model PatchTST, reducing MSE by 6.5% and MAE by 5.8% in multivariate time series forecasting. Code is available at: https://github.com/yyg1282142265/Esiformer/tree/main. |
| title | Less is more: Embracing sparsity and interpolation with Esiformer for time series forecasting |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2410.05726 |