Cover Image

QKCV Attention: Enhancing Time Series Forecasting with Static Categorical Embeddings for Both Lightweight and Pre-trained Foundation Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Hao, Ma, Baojun
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

TimeEmb: A Lightweight Static-Dynamic Disentanglement Framework for Time Series Forecasting
by: Xia, Mingyuan, et al.
Published: (2025)

A Mamba Foundation Model for Time Series Forecasting
by: Ma, Haoyu, et al.
Published: (2024)

TimelyGPT: Extrapolatable Transformer Pre-training for Long-term Time-Series Forecasting in Healthcare
by: Song, Ziyang, et al.
Published: (2023)

Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Series
by: Ekambaram, Vijay, et al.
Published: (2024)

Measuring Pre-training Data Quality without Labels for Time Series Foundation Models
by: Wen, Songkang, et al.
Published: (2024)

Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenization
by: Masserano, Luca, et al.
Published: (2024)

TripCast: Pre-training of Masked 2D Transformers for Trip Time Series Forecasting
by: Liao, Yuhua, et al.
Published: (2024)

Time Series Foundation Models for Process Model Forecasting
by: Yu, Yongbo, et al.
Published: (2025)

Lightweight Time Series Data Valuation on Time Series Foundation Models via In-Context Finetuning
by: Wu, Shunyu, et al.
Published: (2025)

Beyond Static Uncertainty: Modeling Temporal Uncertainty Dynamics for Probabilistic Time Series Forecasting
by: Wang, Yijun, et al.
Published: (2026)

Distilling Time Series Foundation Models for Efficient Forecasting
by: Li, Yuqi, et al.
Published: (2026)

A Time-Series Foundation Model by Universal Delay Embedding
by: Wang, Zijian, et al.
Published: (2025)

Attention as Robust Representation for Time Series Forecasting
by: Niu, PeiSong, et al.
Published: (2024)

TempoPFN: Synthetic Pre-training of Linear RNNs for Zero-shot Time Series Forecasting
by: Moroshan, Vladyslav, et al.
Published: (2025)

Quantifying the Pre-training Dividend: Generative versus Latent Self-Supervised Learning for Time Series Foundation Models
by: Major, Noam, et al.
Published: (2026)

Utilizing Strategic Pre-training to Reduce Overfitting: Baguan -- A Pre-trained Weather Forecasting Model
by: Niu, Peisong, et al.
Published: (2025)

On the Regularization of Learnable Embeddings for Time Series Forecasting
by: Butera, Luca, et al.
Published: (2024)

Non-Stationarity in the Embedding Space of Time Series Foundation Models
by: Choi, Jinmyeong, et al.
Published: (2026)

PSformer: Parameter-efficient Transformer with Segment Attention for Time Series Forecasting
by: Wang, Yanlong, et al.
Published: (2024)

Mantis: Lightweight Calibrated Foundation Model for User-Friendly Time Series Classification
by: Feofanov, Vasilii, et al.
Published: (2025)

Revisiting Attention for Multivariate Time Series Forecasting
by: Wu, Haixiang
Published: (2024)

Are Self-Attentions Effective for Time Series Forecasting?
by: Kim, Dongbin, et al.
Published: (2024)

LENS: Large Pre-trained Transformer for Exploring Financial Time Series Regularities
by: Xu, Yuanjian, et al.
Published: (2024)

SDGF: Fusing Static and Multi-Scale Dynamic Correlations for Multivariate Time Series Forecasting
by: Wang, Shaoxun, et al.
Published: (2025)

OSF: On Pre-training and Scaling of Sleep Foundation Models
by: Shuai, Zitao, et al.
Published: (2026)

TSFMAudit: Data Contamination Auditing in Forecasting Time Series Foundation Models
by: Li, Hongkai, et al.
Published: (2026)

Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting
by: Fu, Xinghong, et al.
Published: (2026)

Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting
by: Rasul, Kashif, et al.
Published: (2023)

WaveMoE: A Wavelet-Enhanced Mixture-of-Experts Foundation Model for Time Series Forecasting
by: Wu, Shunyu, et al.
Published: (2026)

Enhancing Channel-Independent Time Series Forecasting via Cross-Variate Patch Embedding
by: Shin, Donghwa, et al.
Published: (2025)

Assessing the Operational Viability of Foundation Models for Time Series Forecasting
by: Soni, Kavin, et al.
Published: (2026)

TiCT: A Synthetically Pre-Trained Foundation Model for Time Series Classification
by: Yeh, Chin-Chia Michael, et al.
Published: (2025)

Continuous-Time Linear Positional Embedding for Irregular Time Series Forecasting
by: Kim, Byunghyun, et al.
Published: (2024)

DPANet: Dual Pyramid Attention Network for Multivariate Time Series Forecasting
by: Li, Qianyang, et al.
Published: (2025)

Scaling Law for Time Series Forecasting
by: Shi, Jingzhe, et al.
Published: (2024)

StretchTime: Adaptive Time Series Forecasting via Symplectic Attention
by: Kim, Yubin, et al.
Published: (2026)

Goal-Oriented Time-Series Forecasting: Foundation Framework Design
by: Fechete, Luca-Andrei, et al.
Published: (2025)

FAITH: Frequency-domain Attention In Two Horizons for Time Series Forecasting
by: Li, Ruiqi, et al.
Published: (2024)

PENGUIN: Enhancing Transformer with Periodic-Nested Group Attention for Long-term Time Series Forecasting
by: Sun, Tian, et al.
Published: (2025)

PeriodNet: Boosting the Potential of Attention Mechanism for Time Series Forecasting
by: Zhao, Bowen, et al.
Published: (2025)