Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.20167 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918069871312896 |
|---|---|
| author | Li, Fengze Wang, Yue Liu, Yangle Huang, Ming Hong, Dou Ma, Jieming |
| author_facet | Li, Fengze Wang, Yue Liu, Yangle Huang, Ming Hong, Dou Ma, Jieming |
| contents | Multivariate time series forecasting requires models to simultaneously capture variable-wise structural dependencies and generalize across diverse tasks. While structural encoders are effective in modeling feature interactions, they lack the capacity to support semantic-level reasoning or task adaptation. Conversely, large language models (LLMs) possess strong generalization capabilities but remain incompatible with raw time series inputs. This gap limits the development of unified, transferable prediction systems. Therefore, we introduce SEED, a structural encoder for embedding-driven decoding, which integrates four stages: a token-aware encoder for patch extraction, a projection module that aligns patches with language model embeddings, a semantic reprogramming mechanism that maps patches to task-aware prototypes, and a frozen language model for prediction. This modular architecture decouples representation learning from inference, enabling efficient alignment between numerical patterns and semantic reasoning. Empirical results demonstrate that the proposed method achieves consistent improvements over strong baselines, and comparative studies on various datasets confirm SEED's role in addressing the structural-semantic modeling gap. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_20167 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SEED: A Structural Encoder for Embedding-Driven Decoding in Time Series Prediction with LLMs Li, Fengze Wang, Yue Liu, Yangle Huang, Ming Hong, Dou Ma, Jieming Computation and Language Artificial Intelligence Multivariate time series forecasting requires models to simultaneously capture variable-wise structural dependencies and generalize across diverse tasks. While structural encoders are effective in modeling feature interactions, they lack the capacity to support semantic-level reasoning or task adaptation. Conversely, large language models (LLMs) possess strong generalization capabilities but remain incompatible with raw time series inputs. This gap limits the development of unified, transferable prediction systems. Therefore, we introduce SEED, a structural encoder for embedding-driven decoding, which integrates four stages: a token-aware encoder for patch extraction, a projection module that aligns patches with language model embeddings, a semantic reprogramming mechanism that maps patches to task-aware prototypes, and a frozen language model for prediction. This modular architecture decouples representation learning from inference, enabling efficient alignment between numerical patterns and semantic reasoning. Empirical results demonstrate that the proposed method achieves consistent improvements over strong baselines, and comparative studies on various datasets confirm SEED's role in addressing the structural-semantic modeling gap. |
| title | SEED: A Structural Encoder for Embedding-Driven Decoding in Time Series Prediction with LLMs |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2506.20167 |