Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917584721412096 |
|---|---|
| author | Rasul, Kashif Ashok, Arjun Williams, Andrew Robert Ghonia, Hena Bhagwatkar, Rishika Khorasani, Arian Bayazi, Mohammad Javad Darvishi Adamopoulos, George Riachi, Roland Hassen, Nadhir Biloš, Marin Garg, Sahil Schneider, Anderson Chapados, Nicolas Drouin, Alexandre Zantedeschi, Valentina Nevmyvaka, Yuriy Rish, Irina |
| author_facet | Rasul, Kashif Ashok, Arjun Williams, Andrew Robert Ghonia, Hena Bhagwatkar, Rishika Khorasani, Arian Bayazi, Mohammad Javad Darvishi Adamopoulos, George Riachi, Roland Hassen, Nadhir Biloš, Marin Garg, Sahil Schneider, Anderson Chapados, Nicolas Drouin, Alexandre Zantedeschi, Valentina Nevmyvaka, Yuriy Rish, Irina |
| contents | Over the past years, foundation models have caused a paradigm shift in machine learning due to their unprecedented capabilities for zero-shot and few-shot generalization. However, despite the success of foundation models in modalities such as natural language processing and computer vision, the development of foundation models for time series forecasting has lagged behind. We present Lag-Llama, a general-purpose foundation model for univariate probabilistic time series forecasting based on a decoder-only transformer architecture that uses lags as covariates. Lag-Llama is pretrained on a large corpus of diverse time series data from several domains, and demonstrates strong zero-shot generalization capabilities compared to a wide range of forecasting models on downstream datasets across domains. Moreover, when fine-tuned on relatively small fractions of such previously unseen datasets, Lag-Llama achieves state-of-the-art performance, outperforming prior deep learning approaches, emerging as the best general-purpose model on average. Lag-Llama serves as a strong contender to the current state-of-art in time series forecasting and paves the way for future advancements in foundation models tailored to time series data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_08278 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting Rasul, Kashif Ashok, Arjun Williams, Andrew Robert Ghonia, Hena Bhagwatkar, Rishika Khorasani, Arian Bayazi, Mohammad Javad Darvishi Adamopoulos, George Riachi, Roland Hassen, Nadhir Biloš, Marin Garg, Sahil Schneider, Anderson Chapados, Nicolas Drouin, Alexandre Zantedeschi, Valentina Nevmyvaka, Yuriy Rish, Irina Machine Learning Artificial Intelligence Over the past years, foundation models have caused a paradigm shift in machine learning due to their unprecedented capabilities for zero-shot and few-shot generalization. However, despite the success of foundation models in modalities such as natural language processing and computer vision, the development of foundation models for time series forecasting has lagged behind. We present Lag-Llama, a general-purpose foundation model for univariate probabilistic time series forecasting based on a decoder-only transformer architecture that uses lags as covariates. Lag-Llama is pretrained on a large corpus of diverse time series data from several domains, and demonstrates strong zero-shot generalization capabilities compared to a wide range of forecasting models on downstream datasets across domains. Moreover, when fine-tuned on relatively small fractions of such previously unseen datasets, Lag-Llama achieves state-of-the-art performance, outperforming prior deep learning approaches, emerging as the best general-purpose model on average. Lag-Llama serves as a strong contender to the current state-of-art in time series forecasting and paves the way for future advancements in foundation models tailored to time series data. |
| title | Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2310.08278 |