Towards Long-Context Time Series Foundation Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912037259444224 |
|---|---|
| author | Żukowska, Nina Goswami, Mononito Wiliński, Michał Potosnak, Willa Dubrawski, Artur |
| author_facet | Żukowska, Nina Goswami, Mononito Wiliński, Michał Potosnak, Willa Dubrawski, Artur |
| contents | Time series foundation models have shown impressive performance on a variety of tasks, across a wide range of domains, even in zero-shot settings. However, most of these models are designed to handle short univariate time series as an input. This limits their practical use, especially in domains such as healthcare with copious amounts of long and multivariate data with strong temporal and intra-variate dependencies. Our study bridges this gap by cataloging and systematically comparing various context expansion techniques from both language and time series domains, and introducing a novel compressive memory mechanism to allow encoder-only TSFMs to effectively model intra-variate dependencies. We demonstrate the benefits of our approach by imbuing MOMENT, a recent family of multi-task time series foundation models, with the multivariate context. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_13530 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Towards Long-Context Time Series Foundation Models Żukowska, Nina Goswami, Mononito Wiliński, Michał Potosnak, Willa Dubrawski, Artur Machine Learning Time series foundation models have shown impressive performance on a variety of tasks, across a wide range of domains, even in zero-shot settings. However, most of these models are designed to handle short univariate time series as an input. This limits their practical use, especially in domains such as healthcare with copious amounts of long and multivariate data with strong temporal and intra-variate dependencies. Our study bridges this gap by cataloging and systematically comparing various context expansion techniques from both language and time series domains, and introducing a novel compressive memory mechanism to allow encoder-only TSFMs to effectively model intra-variate dependencies. We demonstrate the benefits of our approach by imbuing MOMENT, a recent family of multi-task time series foundation models, with the multivariate context. |
| title | Towards Long-Context Time Series Foundation Models |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2409.13530 |