Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2406.14045 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914327999545344 |
|---|---|
| author | Chuang, Yu-Neng Li, Songchen Yuan, Jiayi Wang, Guanchu Lai, Kwei-Herng Han, Joshua Xu, Zihang Sui, Songyuan Yu, Leisheng Ding, Sirui Chang, Chia-Yuan Reyes, Alfredo Costilla Zha, Daochen Hu, Xia |
| author_facet | Chuang, Yu-Neng Li, Songchen Yuan, Jiayi Wang, Guanchu Lai, Kwei-Herng Han, Joshua Xu, Zihang Sui, Songyuan Yu, Leisheng Ding, Sirui Chang, Chia-Yuan Reyes, Alfredo Costilla Zha, Daochen Hu, Xia |
| contents | Time Series Forecasting (TSF) has long been a challenge in time series analysis. Inspired by the success of Large Language Models (LLMs), researchers are now developing Large Time Series Models (LTSMs)-universal transformer-based models that use autoregressive prediction-to improve TSF. However, training LTSMs on heterogeneous time series data poses unique challenges, including diverse frequencies, dimensions, and patterns across datasets. Recent endeavors have studied and evaluated various design choices aimed at enhancing LTSM training and generalization capabilities. However, these design choices are typically studied and evaluated in isolation and are not benchmarked collectively. In this work, we introduce LTSM-Bundle, a comprehensive toolbox, and benchmark for training LTSMs, spanning pre-processing techniques, model configurations, and dataset configuration. It modularized and benchmarked LTSMs from multiple dimensions, encompassing prompting strategies, tokenization approaches, training paradigms, base model selection, data quantity, and dataset diversity. Furthermore, we combine the most effective design choices identified in our study. Empirical results demonstrate that this combination achieves superior zero-shot and few-shot performances compared to state-of-the-art LTSMs and traditional TSF methods on benchmark datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_14045 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LTSM-Bundle: A Toolbox and Benchmark on Large Language Models for Time Series Forecasting Chuang, Yu-Neng Li, Songchen Yuan, Jiayi Wang, Guanchu Lai, Kwei-Herng Han, Joshua Xu, Zihang Sui, Songyuan Yu, Leisheng Ding, Sirui Chang, Chia-Yuan Reyes, Alfredo Costilla Zha, Daochen Hu, Xia Machine Learning Artificial Intelligence Time Series Forecasting (TSF) has long been a challenge in time series analysis. Inspired by the success of Large Language Models (LLMs), researchers are now developing Large Time Series Models (LTSMs)-universal transformer-based models that use autoregressive prediction-to improve TSF. However, training LTSMs on heterogeneous time series data poses unique challenges, including diverse frequencies, dimensions, and patterns across datasets. Recent endeavors have studied and evaluated various design choices aimed at enhancing LTSM training and generalization capabilities. However, these design choices are typically studied and evaluated in isolation and are not benchmarked collectively. In this work, we introduce LTSM-Bundle, a comprehensive toolbox, and benchmark for training LTSMs, spanning pre-processing techniques, model configurations, and dataset configuration. It modularized and benchmarked LTSMs from multiple dimensions, encompassing prompting strategies, tokenization approaches, training paradigms, base model selection, data quantity, and dataset diversity. Furthermore, we combine the most effective design choices identified in our study. Empirical results demonstrate that this combination achieves superior zero-shot and few-shot performances compared to state-of-the-art LTSMs and traditional TSF methods on benchmark datasets. |
| title | LTSM-Bundle: A Toolbox and Benchmark on Large Language Models for Time Series Forecasting |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2406.14045 |