Quantum-Enhanced Channel Mixing in RWKV Models for Time Series Forecasting
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918040953683968 |
|---|---|
| author | Chen, Chi-Sheng Kuo, En-Jui |
| author_facet | Chen, Chi-Sheng Kuo, En-Jui |
| contents | Recent advancements in neural sequence modeling have led to architectures such as RWKV, which combine recurrent-style time mixing with feedforward channel mixing to enable efficient long-context processing. In this work, we propose QuantumRWKV, a hybrid quantum-classical extension of the RWKV model, where the standard feedforward network (FFN) is partially replaced by a variational quantum circuit (VQC). The quantum component is designed to enhance nonlinear representational capacity while preserving end-to-end differentiability via the PennyLane framework. To assess the impact of quantum enhancements, we conduct a comparative evaluation between QuantumRWKV and its classical counterpart across ten synthetic time-series forecasting tasks, encompassing linear (ARMA), chaotic (Logistic Map), oscillatory (Damped Oscillator, Sine Wave), and regime-switching signals. Our results show that QuantumRWKV outperforms the classical model in 6 out of 10 tasks, particularly excelling in sequences with nonlinear or chaotic dynamics, such as Chaotic Logistic, Noisy Damped Oscillator, Sine Wave, Triangle Wave, Sawtooth, and ARMA. However, it underperforms on tasks involving sharp regime shifts (Piecewise Regime) or smoother periodic patterns (Damped Oscillator, Seasonal Trend, Square Wave). This study provides one of the first systematic comparisons between hybrid quantum-classical and classical recurrent models in temporal domains, highlighting the scenarios where quantum circuits can offer tangible advantages. We conclude with a discussion on architectural trade-offs, such as variance sensitivity in quantum layers, and outline future directions for scaling quantum integration in long-context temporal learning systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_13524 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Quantum-Enhanced Channel Mixing in RWKV Models for Time Series Forecasting Chen, Chi-Sheng Kuo, En-Jui Quantum Physics Recent advancements in neural sequence modeling have led to architectures such as RWKV, which combine recurrent-style time mixing with feedforward channel mixing to enable efficient long-context processing. In this work, we propose QuantumRWKV, a hybrid quantum-classical extension of the RWKV model, where the standard feedforward network (FFN) is partially replaced by a variational quantum circuit (VQC). The quantum component is designed to enhance nonlinear representational capacity while preserving end-to-end differentiability via the PennyLane framework. To assess the impact of quantum enhancements, we conduct a comparative evaluation between QuantumRWKV and its classical counterpart across ten synthetic time-series forecasting tasks, encompassing linear (ARMA), chaotic (Logistic Map), oscillatory (Damped Oscillator, Sine Wave), and regime-switching signals. Our results show that QuantumRWKV outperforms the classical model in 6 out of 10 tasks, particularly excelling in sequences with nonlinear or chaotic dynamics, such as Chaotic Logistic, Noisy Damped Oscillator, Sine Wave, Triangle Wave, Sawtooth, and ARMA. However, it underperforms on tasks involving sharp regime shifts (Piecewise Regime) or smoother periodic patterns (Damped Oscillator, Seasonal Trend, Square Wave). This study provides one of the first systematic comparisons between hybrid quantum-classical and classical recurrent models in temporal domains, highlighting the scenarios where quantum circuits can offer tangible advantages. We conclude with a discussion on architectural trade-offs, such as variance sensitivity in quantum layers, and outline future directions for scaling quantum integration in long-context temporal learning systems. |
| title | Quantum-Enhanced Channel Mixing in RWKV Models for Time Series Forecasting |
| topic | Quantum Physics |
| url | https://arxiv.org/abs/2505.13524 |