Quantum-PEFT: Ultra parameter-efficient fine-tuning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915186199232512 |
|---|---|
| author | Koike-Akino, Toshiaki Tonin, Francesco Wu, Yongtao Wu, Frank Zhengqing Candogan, Leyla Naz Cevher, Volkan |
| author_facet | Koike-Akino, Toshiaki Tonin, Francesco Wu, Yongtao Wu, Frank Zhengqing Candogan, Leyla Naz Cevher, Volkan |
| contents | This paper introduces Quantum-PEFT that leverages quantum computations for parameter-efficient fine-tuning (PEFT). Unlike other additive PEFT methods, such as low-rank adaptation (LoRA), Quantum-PEFT exploits an underlying full-rank yet surprisingly parameter efficient quantum unitary parameterization. With the use of Pauli parameterization, the number of trainable parameters grows only logarithmically with the ambient dimension, as opposed to linearly as in LoRA-based PEFT methods. Quantum-PEFT achieves vanishingly smaller number of trainable parameters than the lowest-rank LoRA as dimensions grow, enhancing parameter efficiency while maintaining a competitive performance. We apply Quantum-PEFT to several transfer learning benchmarks in language and vision, demonstrating significant advantages in parameter efficiency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_05431 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Quantum-PEFT: Ultra parameter-efficient fine-tuning Koike-Akino, Toshiaki Tonin, Francesco Wu, Yongtao Wu, Frank Zhengqing Candogan, Leyla Naz Cevher, Volkan Machine Learning This paper introduces Quantum-PEFT that leverages quantum computations for parameter-efficient fine-tuning (PEFT). Unlike other additive PEFT methods, such as low-rank adaptation (LoRA), Quantum-PEFT exploits an underlying full-rank yet surprisingly parameter efficient quantum unitary parameterization. With the use of Pauli parameterization, the number of trainable parameters grows only logarithmically with the ambient dimension, as opposed to linearly as in LoRA-based PEFT methods. Quantum-PEFT achieves vanishingly smaller number of trainable parameters than the lowest-rank LoRA as dimensions grow, enhancing parameter efficiency while maintaining a competitive performance. We apply Quantum-PEFT to several transfer learning benchmarks in language and vision, demonstrating significant advantages in parameter efficiency. |
| title | Quantum-PEFT: Ultra parameter-efficient fine-tuning |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2503.05431 |