Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.14087 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915544276402176 |
|---|---|
| author | Qiao, Zhongzheng Liu, Chenghao Zhang, Yiming Jin, Ming Pham, Quang Wen, Qingsong Suganthan, P. N. Jiang, Xudong Ramasamy, Savitha |
| author_facet | Qiao, Zhongzheng Liu, Chenghao Zhang, Yiming Jin, Ming Pham, Quang Wen, Qingsong Suganthan, P. N. Jiang, Xudong Ramasamy, Savitha |
| contents | Time series foundation models (TSFMs) demonstrate impressive zero-shot performance for time series forecasting. However, an important yet underexplored challenge is how to effectively finetune TSFMs on specific downstream tasks. While naive finetuning can yield performance gains, we argue that it falls short of fully leveraging TSFMs' capabilities, often resulting in overfitting and suboptimal performance. Given the diverse temporal patterns across sampling scales and the inherent multi-scale forecasting capabilities of TSFMs, we adopt a causal perspective to analyze finetuning process, through which we highlight the critical importance of explicitly modeling multiple scales and reveal the shortcomings of naive approaches. Focusing on encoder-based TSFMs, we propose Multiscale finetuning (MSFT), a simple yet general framework that explicitly integrates multi-scale modeling into the finetuning process. Experimental results on three different backbones (Moirai, Moment and Units) demonstrate that TSFMs finetuned with MSFT not only outperform naive and typical parameter efficient finetuning methods but also surpass state-of-the-art deep learning methods. Codes are available at https://github.com/zqiao11/MSFT. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_14087 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Multi-Scale Finetuning for Encoder-based Time Series Foundation Models Qiao, Zhongzheng Liu, Chenghao Zhang, Yiming Jin, Ming Pham, Quang Wen, Qingsong Suganthan, P. N. Jiang, Xudong Ramasamy, Savitha Machine Learning Time series foundation models (TSFMs) demonstrate impressive zero-shot performance for time series forecasting. However, an important yet underexplored challenge is how to effectively finetune TSFMs on specific downstream tasks. While naive finetuning can yield performance gains, we argue that it falls short of fully leveraging TSFMs' capabilities, often resulting in overfitting and suboptimal performance. Given the diverse temporal patterns across sampling scales and the inherent multi-scale forecasting capabilities of TSFMs, we adopt a causal perspective to analyze finetuning process, through which we highlight the critical importance of explicitly modeling multiple scales and reveal the shortcomings of naive approaches. Focusing on encoder-based TSFMs, we propose Multiscale finetuning (MSFT), a simple yet general framework that explicitly integrates multi-scale modeling into the finetuning process. Experimental results on three different backbones (Moirai, Moment and Units) demonstrate that TSFMs finetuned with MSFT not only outperform naive and typical parameter efficient finetuning methods but also surpass state-of-the-art deep learning methods. Codes are available at https://github.com/zqiao11/MSFT. |
| title | Multi-Scale Finetuning for Encoder-based Time Series Foundation Models |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2506.14087 |