PSformer: Parameter-efficient Transformer with Segment Attention for Time Series Forecasting
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915145157967872 |
|---|---|
| author | Wang, Yanlong Xu, Jian Ma, Fei Huang, Shao-Lun Sun, Danny Dongning Zhang, Xiao-Ping |
| author_facet | Wang, Yanlong Xu, Jian Ma, Fei Huang, Shao-Lun Sun, Danny Dongning Zhang, Xiao-Ping |
| contents | Time series forecasting remains a critical challenge across various domains, often complicated by high-dimensional data and long-term dependencies. This paper presents a novel transformer architecture for time series forecasting, incorporating two key innovations: parameter sharing (PS) and Spatial-Temporal Segment Attention (SegAtt). We also define the time series segment as the concatenation of sequence patches from the same positions across different variables. The proposed model, PSformer, reduces the number of training parameters through the parameter sharing mechanism, thereby improving model efficiency and scalability. The introduction of SegAtt could enhance the capability of capturing local spatio-temporal dependencies by computing attention over the segments, and improve global representation by integrating information across segments. The combination of parameter sharing and SegAtt significantly improves the forecasting performance. Extensive experiments on benchmark datasets demonstrate that PSformer outperforms popular baselines and other transformer-based approaches in terms of accuracy and scalability, establishing itself as an accurate and scalable tool for time series forecasting. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_01419 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PSformer: Parameter-efficient Transformer with Segment Attention for Time Series Forecasting Wang, Yanlong Xu, Jian Ma, Fei Huang, Shao-Lun Sun, Danny Dongning Zhang, Xiao-Ping Machine Learning Artificial Intelligence Time series forecasting remains a critical challenge across various domains, often complicated by high-dimensional data and long-term dependencies. This paper presents a novel transformer architecture for time series forecasting, incorporating two key innovations: parameter sharing (PS) and Spatial-Temporal Segment Attention (SegAtt). We also define the time series segment as the concatenation of sequence patches from the same positions across different variables. The proposed model, PSformer, reduces the number of training parameters through the parameter sharing mechanism, thereby improving model efficiency and scalability. The introduction of SegAtt could enhance the capability of capturing local spatio-temporal dependencies by computing attention over the segments, and improve global representation by integrating information across segments. The combination of parameter sharing and SegAtt significantly improves the forecasting performance. Extensive experiments on benchmark datasets demonstrate that PSformer outperforms popular baselines and other transformer-based approaches in terms of accuracy and scalability, establishing itself as an accurate and scalable tool for time series forecasting. |
| title | PSformer: Parameter-efficient Transformer with Segment Attention for Time Series Forecasting |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2411.01419 |