Giving Sensors a Voice: Multimodal JEPA for Semantic Time-Series Embeddings
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911733702983680 |
|---|---|
| author | Dutta, Utsav Pastrana, Gerardo Pakazad, Sina Khoshfetrat Ohlsson, Henrik |
| author_facet | Dutta, Utsav Pastrana, Gerardo Pakazad, Sina Khoshfetrat Ohlsson, Henrik |
| contents | Transformer-based architectures have advanced sequence modeling in language and vision, yet general-purpose representation learning for heterogeneous multivariate time series remains underexplored. We introduce CHARM (Channel-Aware Representation Model), which incorporates channel-level textual descriptions into a Transformer encoder equivariant to channel order. CHARM is trained with a Joint Embedding Predictive Architecture (JEPA) and a novel loss promoting informative, temporally stable embeddings; latent-space prediction encourages robustness to sensor noise while description-aware gating provides interpretability through learned inter-channel relationships. Across anomaly detection, classification, and short- and long-term forecasting, the learned embeddings achieve strong performance using only a linear probe. Performance is driven primarily by the JEPA objective and conditioning architecture, with text descriptions serving as channel identifiers for cross-dataset generalization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_31580 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Giving Sensors a Voice: Multimodal JEPA for Semantic Time-Series Embeddings Dutta, Utsav Pastrana, Gerardo Pakazad, Sina Khoshfetrat Ohlsson, Henrik Machine Learning I.2.6; I.5.1 Transformer-based architectures have advanced sequence modeling in language and vision, yet general-purpose representation learning for heterogeneous multivariate time series remains underexplored. We introduce CHARM (Channel-Aware Representation Model), which incorporates channel-level textual descriptions into a Transformer encoder equivariant to channel order. CHARM is trained with a Joint Embedding Predictive Architecture (JEPA) and a novel loss promoting informative, temporally stable embeddings; latent-space prediction encourages robustness to sensor noise while description-aware gating provides interpretability through learned inter-channel relationships. Across anomaly detection, classification, and short- and long-term forecasting, the learned embeddings achieve strong performance using only a linear probe. Performance is driven primarily by the JEPA objective and conditioning architecture, with text descriptions serving as channel identifiers for cross-dataset generalization. |
| title | Giving Sensors a Voice: Multimodal JEPA for Semantic Time-Series Embeddings |
| topic | Machine Learning I.2.6; I.5.1 |
| url | https://arxiv.org/abs/2605.31580 |