QR-LoRA: Efficient and Disentangled Fine-tuning via QR Decomposition for Customized Generation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908463389474816 |
|---|---|
| author | Yang, Jiahui Ma, Yongjia Di, Donglin Li, Hao Chen, Wei Xie, Yan Cui, Jianxun Yang, Xun Zuo, Wangmeng |
| author_facet | Yang, Jiahui Ma, Yongjia Di, Donglin Li, Hao Chen, Wei Xie, Yan Cui, Jianxun Yang, Xun Zuo, Wangmeng |
| contents | Existing text-to-image models often rely on parameter fine-tuning techniques such as Low-Rank Adaptation (LoRA) to customize visual attributes. However, when combining multiple LoRA models for content-style fusion tasks, unstructured modifications of weight matrices often lead to undesired feature entanglement between content and style attributes. We propose QR-LoRA, a novel fine-tuning framework leveraging QR decomposition for structured parameter updates that effectively separate visual attributes. Our key insight is that the orthogonal Q matrix naturally minimizes interference between different visual features, while the upper triangular R matrix efficiently encodes attribute-specific transformations. Our approach fixes both Q and R matrices while only training an additional task-specific $ΔR$ matrix. This structured design reduces trainable parameters to half of conventional LoRA methods and supports effective merging of multiple adaptations without cross-contamination due to the strong disentanglement properties between $ΔR$ matrices. Experiments demonstrate that QR-LoRA achieves superior disentanglement in content-style fusion tasks, establishing a new paradigm for parameter-efficient, disentangled fine-tuning in generative models. The project page is available at: https://luna-ai-lab.github.io/QR-LoRA/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_04599 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | QR-LoRA: Efficient and Disentangled Fine-tuning via QR Decomposition for Customized Generation Yang, Jiahui Ma, Yongjia Di, Donglin Li, Hao Chen, Wei Xie, Yan Cui, Jianxun Yang, Xun Zuo, Wangmeng Computer Vision and Pattern Recognition Existing text-to-image models often rely on parameter fine-tuning techniques such as Low-Rank Adaptation (LoRA) to customize visual attributes. However, when combining multiple LoRA models for content-style fusion tasks, unstructured modifications of weight matrices often lead to undesired feature entanglement between content and style attributes. We propose QR-LoRA, a novel fine-tuning framework leveraging QR decomposition for structured parameter updates that effectively separate visual attributes. Our key insight is that the orthogonal Q matrix naturally minimizes interference between different visual features, while the upper triangular R matrix efficiently encodes attribute-specific transformations. Our approach fixes both Q and R matrices while only training an additional task-specific $ΔR$ matrix. This structured design reduces trainable parameters to half of conventional LoRA methods and supports effective merging of multiple adaptations without cross-contamination due to the strong disentanglement properties between $ΔR$ matrices. Experiments demonstrate that QR-LoRA achieves superior disentanglement in content-style fusion tasks, establishing a new paradigm for parameter-efficient, disentangled fine-tuning in generative models. The project page is available at: https://luna-ai-lab.github.io/QR-LoRA/. |
| title | QR-LoRA: Efficient and Disentangled Fine-tuning via QR Decomposition for Customized Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2507.04599 |