Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.14543 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908372343717888 |
|---|---|
| author | Dutta, Utsav Pakazad, Sina Khoshfetrat Ohlsson, Henrik |
| author_facet | Dutta, Utsav Pakazad, Sina Khoshfetrat Ohlsson, Henrik |
| contents | Traditional time series models are task-specific and often depend on dataset-specific training and extensive feature engineering. While Transformer-based architectures have improved scalability, foundation models, commonplace in text, vision, and audio, remain under-explored for time series and are largely restricted to forecasting. We introduce $\textbf{CHARM}$, a foundation embedding model for multivariate time series that learns shared, transferable, and domain-aware representations. To address the unique difficulties of time series foundation learning, $\textbf{CHARM}$ incorporates architectural innovations that integrate channel-level textual descriptions while remaining invariant to channel order. The model is trained using a Joint Embedding Predictive Architecture (JEPA), with novel augmentation schemes and a loss function designed to improve interpretability and training stability. Our $7$M-parameter model achieves state-of-the-art performance across diverse downstream tasks, setting a new benchmark for time series representation learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_14543 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Time to Embed: Unlocking Foundation Models for Time Series with Channel Descriptions Dutta, Utsav Pakazad, Sina Khoshfetrat Ohlsson, Henrik Machine Learning Traditional time series models are task-specific and often depend on dataset-specific training and extensive feature engineering. While Transformer-based architectures have improved scalability, foundation models, commonplace in text, vision, and audio, remain under-explored for time series and are largely restricted to forecasting. We introduce $\textbf{CHARM}$, a foundation embedding model for multivariate time series that learns shared, transferable, and domain-aware representations. To address the unique difficulties of time series foundation learning, $\textbf{CHARM}$ incorporates architectural innovations that integrate channel-level textual descriptions while remaining invariant to channel order. The model is trained using a Joint Embedding Predictive Architecture (JEPA), with novel augmentation schemes and a loss function designed to improve interpretability and training stability. Our $7$M-parameter model achieves state-of-the-art performance across diverse downstream tasks, setting a new benchmark for time series representation learning. |
| title | Time to Embed: Unlocking Foundation Models for Time Series with Channel Descriptions |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2505.14543 |