Disentangled Parameter-Efficient Linear Model for Long-Term Time Series Forecasting
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911432522596352 |
|---|---|
| author | Zhao, Yuang Li, Tianyu Chen, Jiadong Ye, Shenrong Jiang, Fuxin Gao, Xiaofeng |
| author_facet | Zhao, Yuang Li, Tianyu Chen, Jiadong Ye, Shenrong Jiang, Fuxin Gao, Xiaofeng |
| contents | Long-term Time Series Forecasting (LTSF) is crucial across various domains, but complex deep models like Transformers are often prone to overfitting on extended sequences. Linear Fully Connected models have emerged as a powerful alternative, achieving competitive results with fewer parameters. However, their reliance on a single, monolithic weight matrix leads to quadratic parameter redundancy and an entanglement of temporal and frequential properties. To address this, we propose DiPE-Linear, a novel model that disentangles this monolithic mapping into a sequence of specialized, parameter-efficient modules. DiPE-Linear features three core components: Static Frequential Attention to prioritize critical frequencies, Static Time Attention to focus on key time steps, and Independent Frequential Mapping to independently process frequency components. A Low-rank Weight Sharing policy further enhances efficiency for multivariate data. This disentangled architecture collectively reduces parameter complexity from quadratic to linear and computational complexity to log-linear. Experiments on real-world datasets show that DiPE-Linear delivers state-of-the-art performance with significantly fewer parameters, establishing a new and highly efficient baseline for LTSF. Our code is available at https://github.com/wintertee/DiPE-Linear/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_17257 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Disentangled Parameter-Efficient Linear Model for Long-Term Time Series Forecasting Zhao, Yuang Li, Tianyu Chen, Jiadong Ye, Shenrong Jiang, Fuxin Gao, Xiaofeng Machine Learning Artificial Intelligence Long-term Time Series Forecasting (LTSF) is crucial across various domains, but complex deep models like Transformers are often prone to overfitting on extended sequences. Linear Fully Connected models have emerged as a powerful alternative, achieving competitive results with fewer parameters. However, their reliance on a single, monolithic weight matrix leads to quadratic parameter redundancy and an entanglement of temporal and frequential properties. To address this, we propose DiPE-Linear, a novel model that disentangles this monolithic mapping into a sequence of specialized, parameter-efficient modules. DiPE-Linear features three core components: Static Frequential Attention to prioritize critical frequencies, Static Time Attention to focus on key time steps, and Independent Frequential Mapping to independently process frequency components. A Low-rank Weight Sharing policy further enhances efficiency for multivariate data. This disentangled architecture collectively reduces parameter complexity from quadratic to linear and computational complexity to log-linear. Experiments on real-world datasets show that DiPE-Linear delivers state-of-the-art performance with significantly fewer parameters, establishing a new and highly efficient baseline for LTSF. Our code is available at https://github.com/wintertee/DiPE-Linear/ |
| title | Disentangled Parameter-Efficient Linear Model for Long-Term Time Series Forecasting |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2411.17257 |