TiCT: A Synthetically Pre-Trained Foundation Model for Time Series Classification
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914171165081600 |
|---|---|
| author | Yeh, Chin-Chia Michael Saini, Uday Singh Wang, Junpeng Dai, Xin Fan, Xiran Sun, Jiarui Fan, Yujie Zheng, Yan |
| author_facet | Yeh, Chin-Chia Michael Saini, Uday Singh Wang, Junpeng Dai, Xin Fan, Xiran Sun, Jiarui Fan, Yujie Zheng, Yan |
| contents | The ubiquity of time series data creates a strong demand for general-purpose foundation models, yet developing them for classification remains a significant challenge, largely due to the high cost of labeled data. Foundation models capable of in-context learning (ICL) offer a powerful solution, adapting to new tasks with minimal examples and reducing the need for extensive retraining. However, prior work on large-scale time series models has predominantly focused on forecasting, leaving a critical gap for versatile, fine-tuning-free classification. To address this, we introduce TiCT (Time-series in-Context Transformer), a transformer-based model pre-trained exclusively on synthetic data to perform in-context classification. We make two primary technical contributions: 1) a novel architecture featuring a scalable bit-based label encoding and a special output attention mechanism to handle an arbitrary number of classes; and 2) a synthetic pre-training framework that combines a Mixup-inspired process with data augmentation to foster generalization and noise invariance. Extensive evaluations on the UCR Archive show that TiCT achieves competitive performance against state-of-the-art supervised methods. Crucially, this is accomplished using only in-context examples at inference time, without updating a single model weight. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_19694 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TiCT: A Synthetically Pre-Trained Foundation Model for Time Series Classification Yeh, Chin-Chia Michael Saini, Uday Singh Wang, Junpeng Dai, Xin Fan, Xiran Sun, Jiarui Fan, Yujie Zheng, Yan Machine Learning Artificial Intelligence The ubiquity of time series data creates a strong demand for general-purpose foundation models, yet developing them for classification remains a significant challenge, largely due to the high cost of labeled data. Foundation models capable of in-context learning (ICL) offer a powerful solution, adapting to new tasks with minimal examples and reducing the need for extensive retraining. However, prior work on large-scale time series models has predominantly focused on forecasting, leaving a critical gap for versatile, fine-tuning-free classification. To address this, we introduce TiCT (Time-series in-Context Transformer), a transformer-based model pre-trained exclusively on synthetic data to perform in-context classification. We make two primary technical contributions: 1) a novel architecture featuring a scalable bit-based label encoding and a special output attention mechanism to handle an arbitrary number of classes; and 2) a synthetic pre-training framework that combines a Mixup-inspired process with data augmentation to foster generalization and noise invariance. Extensive evaluations on the UCR Archive show that TiCT achieves competitive performance against state-of-the-art supervised methods. Crucially, this is accomplished using only in-context examples at inference time, without updating a single model weight. |
| title | TiCT: A Synthetically Pre-Trained Foundation Model for Time Series Classification |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2511.19694 |