TS-HTFA: Advancing Time Series Forecasting via Hierarchical Text-Free Alignment with Large Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912179636142080 |
|---|---|
| author | Wang, Pengfei Zheng, Huanran Xu, Qi'ao Dai, Silong Wang, Yiqiao Yue, Wenjing Zhu, Wei Qian, Tianwen Wang, Xiaoling |
| author_facet | Wang, Pengfei Zheng, Huanran Xu, Qi'ao Dai, Silong Wang, Yiqiao Yue, Wenjing Zhu, Wei Qian, Tianwen Wang, Xiaoling |
| contents | Given the significant potential of large language models (LLMs) in sequence modeling, emerging studies have begun applying them to time-series forecasting. Despite notable progress, existing methods still face two critical challenges: 1) their reliance on large amounts of paired text data, limiting the model applicability, and 2) a substantial modality gap between text and time series, leading to insufficient alignment and suboptimal performance. In this paper, we introduce \textbf{H}ierarchical \textbf{T}ext-\textbf{F}ree \textbf{A}lignment (\textbf{TS-HTFA}), a novel method that leverages hierarchical alignment to fully exploit the representation capacity of LLMs while eliminating the dependence on text data. Specifically, we replace paired text data with adaptive virtual text based on QR decomposition word embeddings and learnable prompt. Furthermore, we establish comprehensive cross-modal alignment at three levels: input, feature, and output. Extensive experiments on multiple time-series benchmarks demonstrate that HTFA achieves state-of-the-art performance, significantly improving prediction accuracy and generalization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_14978 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | TS-HTFA: Advancing Time Series Forecasting via Hierarchical Text-Free Alignment with Large Language Models Wang, Pengfei Zheng, Huanran Xu, Qi'ao Dai, Silong Wang, Yiqiao Yue, Wenjing Zhu, Wei Qian, Tianwen Wang, Xiaoling Artificial Intelligence Given the significant potential of large language models (LLMs) in sequence modeling, emerging studies have begun applying them to time-series forecasting. Despite notable progress, existing methods still face two critical challenges: 1) their reliance on large amounts of paired text data, limiting the model applicability, and 2) a substantial modality gap between text and time series, leading to insufficient alignment and suboptimal performance. In this paper, we introduce \textbf{H}ierarchical \textbf{T}ext-\textbf{F}ree \textbf{A}lignment (\textbf{TS-HTFA}), a novel method that leverages hierarchical alignment to fully exploit the representation capacity of LLMs while eliminating the dependence on text data. Specifically, we replace paired text data with adaptive virtual text based on QR decomposition word embeddings and learnable prompt. Furthermore, we establish comprehensive cross-modal alignment at three levels: input, feature, and output. Extensive experiments on multiple time-series benchmarks demonstrate that HTFA achieves state-of-the-art performance, significantly improving prediction accuracy and generalization. |
| title | TS-HTFA: Advancing Time Series Forecasting via Hierarchical Text-Free Alignment with Large Language Models |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2409.14978 |