Quantum Large Language Models via Tensor Network Disentanglers
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916449392525312 |
|---|---|
| author | Aizpurua, Borja Jahromi, Saeed S. Singh, Sukhbinder Orus, Roman |
| author_facet | Aizpurua, Borja Jahromi, Saeed S. Singh, Sukhbinder Orus, Roman |
| contents | We propose a method to enhance the performance of Large Language Models (LLMs) by integrating quantum computing and quantum-inspired techniques. Specifically, our approach involves replacing the weight matrices in the Self-Attention and Multi-layer Perceptron layers with a combination of two variational quantum circuits and a quantum-inspired tensor network, such as a Matrix Product Operator (MPO). This substitution enables the reproduction of classical LLM functionality by decomposing weight matrices through the application of tensor network disentanglers and MPOs, leveraging well-established tensor network techniques. By incorporating more complex and deeper quantum circuits, along with increasing the bond dimensions of the MPOs, our method captures additional correlations within the quantum-enhanced LLM, leading to improved accuracy beyond classical models while maintaining low memory overhead. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_17397 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Quantum Large Language Models via Tensor Network Disentanglers Aizpurua, Borja Jahromi, Saeed S. Singh, Sukhbinder Orus, Roman Quantum Physics Artificial Intelligence Machine Learning We propose a method to enhance the performance of Large Language Models (LLMs) by integrating quantum computing and quantum-inspired techniques. Specifically, our approach involves replacing the weight matrices in the Self-Attention and Multi-layer Perceptron layers with a combination of two variational quantum circuits and a quantum-inspired tensor network, such as a Matrix Product Operator (MPO). This substitution enables the reproduction of classical LLM functionality by decomposing weight matrices through the application of tensor network disentanglers and MPOs, leveraging well-established tensor network techniques. By incorporating more complex and deeper quantum circuits, along with increasing the bond dimensions of the MPOs, our method captures additional correlations within the quantum-enhanced LLM, leading to improved accuracy beyond classical models while maintaining low memory overhead. |
| title | Quantum Large Language Models via Tensor Network Disentanglers |
| topic | Quantum Physics Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2410.17397 |