Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.09119 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912621651820544 |
|---|---|
| author | Luo, Jie Kulcsar, Jeremy Chen, Xueyin Giaconi, Giulio Korpas, Georgios |
| author_facet | Luo, Jie Kulcsar, Jeremy Chen, Xueyin Giaconi, Giulio Korpas, Georgios |
| contents | Quantum circuits embed data in a Hilbert space whose dimensionality grows exponentially with the number of qubits, allowing even shallow parameterised quantum circuits (PQCs) to represent highly-correlated probability distributions that are costly for classical networks to capture. Reinforcement-learning (RL) agents, which must reason over long-horizon, continuous-control tasks, stand to benefit from this expressive quantum feature space, but only if the quantum layers can be trained jointly with the surrounding deep-neural components. Current gradient-estimation techniques (e.g., parameter-shift rule) make such hybrid training impractical for realistic RL workloads, because every gradient step requires a prohibitive number of circuit evaluations and thus erodes the potential quantum advantage. We introduce qtDNN, a tangential surrogate that locally approximates a PQC with a small differentiable network trained on-the-fly from the same minibatch. Embedding qtDNN inside the computation graph yields scalable batch gradients while keeping the original quantum layer for inference. Building on qtDNN we design hDQNN-TD3, a hybrid deep quantum neural network for continuous-control reinforcement learning based on the TD3 architecture, which matches or exceeds state-of-the-art classical performance on popular benchmarks. The method opens a path toward applying hybrid quantum models to large-scale RL and other gradient-intensive machine-learning tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_09119 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Training Hybrid Deep Quantum Neural Network for Efficient Reinforcement Learning Luo, Jie Kulcsar, Jeremy Chen, Xueyin Giaconi, Giulio Korpas, Georgios Quantum Physics Quantum circuits embed data in a Hilbert space whose dimensionality grows exponentially with the number of qubits, allowing even shallow parameterised quantum circuits (PQCs) to represent highly-correlated probability distributions that are costly for classical networks to capture. Reinforcement-learning (RL) agents, which must reason over long-horizon, continuous-control tasks, stand to benefit from this expressive quantum feature space, but only if the quantum layers can be trained jointly with the surrounding deep-neural components. Current gradient-estimation techniques (e.g., parameter-shift rule) make such hybrid training impractical for realistic RL workloads, because every gradient step requires a prohibitive number of circuit evaluations and thus erodes the potential quantum advantage. We introduce qtDNN, a tangential surrogate that locally approximates a PQC with a small differentiable network trained on-the-fly from the same minibatch. Embedding qtDNN inside the computation graph yields scalable batch gradients while keeping the original quantum layer for inference. Building on qtDNN we design hDQNN-TD3, a hybrid deep quantum neural network for continuous-control reinforcement learning based on the TD3 architecture, which matches or exceeds state-of-the-art classical performance on popular benchmarks. The method opens a path toward applying hybrid quantum models to large-scale RL and other gradient-intensive machine-learning tasks. |
| title | Training Hybrid Deep Quantum Neural Network for Efficient Reinforcement Learning |
| topic | Quantum Physics |
| url | https://arxiv.org/abs/2503.09119 |