GTM: A General Time-series Model for Enhanced Representation Learning of Time-Series Data
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910049875525632 |
|---|---|
| author | He, Cheng Huang, Xu Jiang, Gangwei Li, Zhaoyi Lian, Defu Xie, Hong Chen, Enhong Liang, Xijie Zheng, Zengrong Lee, Patrick P. C. |
| author_facet | He, Cheng Huang, Xu Jiang, Gangwei Li, Zhaoyi Lian, Defu Xie, Hong Chen, Enhong Liang, Xijie Zheng, Zengrong Lee, Patrick P. C. |
| contents | Despite recent progress in time-series foundation models, challenges persist in improving representation learning and adapting to diverse downstream tasks. We introduce a General Time-series Model (GTM), which advances representation learning via a novel frequency-domain attention mechanism that captures time-granularity-aware features, an aspect underexplored in prior research. We further propose a novel pre-training strategy that unifies reconstruction and autoregressive objectives through a hybrid masking mechanism. Our pre-training strategy, combined with 2D positional encoding and span shuffling, enhances the robustness and generalization of representations. GTM is established as the first generative-task-agnostic model for time-series analysis, enabling seamless adaptation to various generative tasks without any task-specific modifications. Extensive experiments demonstrate that GTM consistently outperforms SOTA models on various generative tasks and achieves strong classification results with minimal adaptation. Furthermore, GTM exhibits clear scaling behavior, with accuracy improving as model size and pre-training data increase. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_03264 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | GTM: A General Time-series Model for Enhanced Representation Learning of Time-Series Data He, Cheng Huang, Xu Jiang, Gangwei Li, Zhaoyi Lian, Defu Xie, Hong Chen, Enhong Liang, Xijie Zheng, Zengrong Lee, Patrick P. C. Machine Learning Despite recent progress in time-series foundation models, challenges persist in improving representation learning and adapting to diverse downstream tasks. We introduce a General Time-series Model (GTM), which advances representation learning via a novel frequency-domain attention mechanism that captures time-granularity-aware features, an aspect underexplored in prior research. We further propose a novel pre-training strategy that unifies reconstruction and autoregressive objectives through a hybrid masking mechanism. Our pre-training strategy, combined with 2D positional encoding and span shuffling, enhances the robustness and generalization of representations. GTM is established as the first generative-task-agnostic model for time-series analysis, enabling seamless adaptation to various generative tasks without any task-specific modifications. Extensive experiments demonstrate that GTM consistently outperforms SOTA models on various generative tasks and achieves strong classification results with minimal adaptation. Furthermore, GTM exhibits clear scaling behavior, with accuracy improving as model size and pre-training data increase. |
| title | GTM: A General Time-series Model for Enhanced Representation Learning of Time-Series Data |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2502.03264 |