Using Pre-trained LLMs for Multivariate Time Series Forecasting
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915100500164608 |
|---|---|
| author | Wolff, Malcolm L. Yang, Shenghao Torkkola, Kari Mahoney, Michael W. |
| author_facet | Wolff, Malcolm L. Yang, Shenghao Torkkola, Kari Mahoney, Michael W. |
| contents | Pre-trained Large Language Models (LLMs) encapsulate large amounts of knowledge and take enormous amounts of compute to train. We make use of this resource, together with the observation that LLMs are able to transfer knowledge and performance from one domain or even modality to another seemingly-unrelated area, to help with multivariate demand time series forecasting. Attention in transformer-based methods requires something worth attending to -- more than just samples of a time-series. We explore different methods to map multivariate input time series into the LLM token embedding space. In particular, our novel multivariate patching strategy to embed time series features into decoder-only pre-trained Transformers produces results competitive with state-of-the-art time series forecasting models. We also use recently-developed weight-based diagnostics to validate our findings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_06386 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Using Pre-trained LLMs for Multivariate Time Series Forecasting Wolff, Malcolm L. Yang, Shenghao Torkkola, Kari Mahoney, Michael W. Machine Learning Computation and Language Pre-trained Large Language Models (LLMs) encapsulate large amounts of knowledge and take enormous amounts of compute to train. We make use of this resource, together with the observation that LLMs are able to transfer knowledge and performance from one domain or even modality to another seemingly-unrelated area, to help with multivariate demand time series forecasting. Attention in transformer-based methods requires something worth attending to -- more than just samples of a time-series. We explore different methods to map multivariate input time series into the LLM token embedding space. In particular, our novel multivariate patching strategy to embed time series features into decoder-only pre-trained Transformers produces results competitive with state-of-the-art time series forecasting models. We also use recently-developed weight-based diagnostics to validate our findings. |
| title | Using Pre-trained LLMs for Multivariate Time Series Forecasting |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2501.06386 |