TSFM-Bench: A Comprehensive and Unified Benchmark of Foundation Models for Time Series Forecasting
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916791340498944 |
|---|---|
| author | Li, Zhe Qiu, Xiangfei Chen, Peng Wang, Yihang Cheng, Hanyin Shu, Yang Hu, Jilin Guo, Chenjuan Zhou, Aoying Jensen, Christian S. Yang, Bin |
| author_facet | Li, Zhe Qiu, Xiangfei Chen, Peng Wang, Yihang Cheng, Hanyin Shu, Yang Hu, Jilin Guo, Chenjuan Zhou, Aoying Jensen, Christian S. Yang, Bin |
| contents | Time Series Forecasting (TSF) is key functionality in numerous fields, such as financial investment, weather services, and energy management. Although increasingly capable TSF methods occur, many of them require domain-specific data collection and model training and do not generalize well when applied in other domains. Time Series Foundation Models (TSFMs) that are pre-trained on massive heterogeneous time series data aim to overcome these limitations. The prospects for generalizability have spurred the development of a new generation of TSFMs. This study proposes a benchmark, TSFM-Bench, to facilitate comprehensive and unified evaluation of TSFMs. TSFM-Bench covers a wide range of TSFMs, including those based on large language models and those pre-trained on time series data. TSFM-Bench supports multiple forecasting scenarios, including zero-shot, few-shot, and full-shot, enabling assessment across the full range of adaptation strategies. TSFM-Bench also provides a standardized experimental protocols for critical evaluation processes such as dataset splitting, loading, normalization, and few-shot sampling, facilitating consistency and fairness. We report on an extensive evaluation of TSFMs across a diverse range of datasets spanning multiple domains and exhibiting varied statistical characteristics. Specifically, we identify pros and cons and inherent limitations of existing TSFMs, and we propose potential directions for new model designs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_11802 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | TSFM-Bench: A Comprehensive and Unified Benchmark of Foundation Models for Time Series Forecasting Li, Zhe Qiu, Xiangfei Chen, Peng Wang, Yihang Cheng, Hanyin Shu, Yang Hu, Jilin Guo, Chenjuan Zhou, Aoying Jensen, Christian S. Yang, Bin Machine Learning Time Series Forecasting (TSF) is key functionality in numerous fields, such as financial investment, weather services, and energy management. Although increasingly capable TSF methods occur, many of them require domain-specific data collection and model training and do not generalize well when applied in other domains. Time Series Foundation Models (TSFMs) that are pre-trained on massive heterogeneous time series data aim to overcome these limitations. The prospects for generalizability have spurred the development of a new generation of TSFMs. This study proposes a benchmark, TSFM-Bench, to facilitate comprehensive and unified evaluation of TSFMs. TSFM-Bench covers a wide range of TSFMs, including those based on large language models and those pre-trained on time series data. TSFM-Bench supports multiple forecasting scenarios, including zero-shot, few-shot, and full-shot, enabling assessment across the full range of adaptation strategies. TSFM-Bench also provides a standardized experimental protocols for critical evaluation processes such as dataset splitting, loading, normalization, and few-shot sampling, facilitating consistency and fairness. We report on an extensive evaluation of TSFMs across a diverse range of datasets spanning multiple domains and exhibiting varied statistical characteristics. Specifically, we identify pros and cons and inherent limitations of existing TSFMs, and we propose potential directions for new model designs. |
| title | TSFM-Bench: A Comprehensive and Unified Benchmark of Foundation Models for Time Series Forecasting |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2410.11802 |