Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.14642 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909794986622976 |
|---|---|
| author | Wu, Yuemin Wu, Zhongze Su, Xiu Yang, Feng Xu, Hongyan Lin, Xi Huang, Wenti You, Shan Xu, Chang |
| author_facet | Wu, Yuemin Wu, Zhongze Su, Xiu Yang, Feng Xu, Hongyan Lin, Xi Huang, Wenti You, Shan Xu, Chang |
| contents | Modeling dynamic temporal dependencies is a critical challenge in time series pre-training, which evolve due to distribution shifts and multi-scale patterns. This temporal variability severely impairs the generalization of pre-trained models to downstream tasks. Existing frameworks fail to capture the complex interactions of short- and long-term dependencies, making them susceptible to spurious correlations that degrade generalization. To address these limitations, we propose DeCoP, a Dependency Controlled Pre-training framework that explicitly models dynamic, multi-scale dependencies by simulating evolving inter-patch dependencies. At the input level, DeCoP introduces Instance-wise Patch Normalization (IPN) to mitigate distributional shifts while preserving the unique characteristics of each patch, creating a robust foundation for representation learning. At the latent level, a hierarchical Dependency Controlled Learning (DCL) strategy explicitly models inter-patch dependencies across multiple temporal scales, with an Instance-level Contrastive Module (ICM) enhances global generalization by learning instance-discriminative representations from time-invariant positive pairs. DeCoP achieves state-of-the-art results on ten datasets with lower computing resources, improving MSE by 3% on ETTh1 over PatchTST using only 37% of the FLOPs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_14642 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DeCoP: Enhancing Self-Supervised Time Series Representation with Dependency Controlled Pre-training Wu, Yuemin Wu, Zhongze Su, Xiu Yang, Feng Xu, Hongyan Lin, Xi Huang, Wenti You, Shan Xu, Chang Machine Learning Artificial Intelligence Modeling dynamic temporal dependencies is a critical challenge in time series pre-training, which evolve due to distribution shifts and multi-scale patterns. This temporal variability severely impairs the generalization of pre-trained models to downstream tasks. Existing frameworks fail to capture the complex interactions of short- and long-term dependencies, making them susceptible to spurious correlations that degrade generalization. To address these limitations, we propose DeCoP, a Dependency Controlled Pre-training framework that explicitly models dynamic, multi-scale dependencies by simulating evolving inter-patch dependencies. At the input level, DeCoP introduces Instance-wise Patch Normalization (IPN) to mitigate distributional shifts while preserving the unique characteristics of each patch, creating a robust foundation for representation learning. At the latent level, a hierarchical Dependency Controlled Learning (DCL) strategy explicitly models inter-patch dependencies across multiple temporal scales, with an Instance-level Contrastive Module (ICM) enhances global generalization by learning instance-discriminative representations from time-invariant positive pairs. DeCoP achieves state-of-the-art results on ten datasets with lower computing resources, improving MSE by 3% on ETTh1 over PatchTST using only 37% of the FLOPs. |
| title | DeCoP: Enhancing Self-Supervised Time Series Representation with Dependency Controlled Pre-training |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2509.14642 |