Exploring Sparsity for Parameter Efficient Fine Tuning Using Wavelets
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866918045249699840 |
|---|---|
| author | Bilican, Ahmet Yılmaz, M. Akın Tekalp, A. Murat Cinbiş, R. Gökberk |
| author_facet | Bilican, Ahmet Yılmaz, M. Akın Tekalp, A. Murat Cinbiş, R. Gökberk |
| contents | Efficiently adapting large foundation models is critical, especially with tight compute and memory budgets. Parameter-Efficient Fine-Tuning (PEFT) methods such as LoRA offer limited granularity and effectiveness in few-parameter regimes. We propose Wavelet Fine-Tuning (WaveFT), a novel PEFT method that learns highly sparse updates in the wavelet domain of residual matrices. WaveFT allows precise control of trainable parameters, offering fine-grained capacity adjustment and excelling with remarkably low parameter count, potentially far fewer than LoRA's minimum, ideal for extreme parameter-efficient scenarios. Evaluated on personalized text-to-image generation using Stable Diffusion XL as baseline, WaveFT significantly outperforms LoRA and other PEFT methods, especially at low parameter counts; achieving superior subject fidelity, prompt alignment, and image diversity. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_12532 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Exploring Sparsity for Parameter Efficient Fine Tuning Using Wavelets Bilican, Ahmet Yılmaz, M. Akın Tekalp, A. Murat Cinbiş, R. Gökberk Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Image and Video Processing Signal Processing Efficiently adapting large foundation models is critical, especially with tight compute and memory budgets. Parameter-Efficient Fine-Tuning (PEFT) methods such as LoRA offer limited granularity and effectiveness in few-parameter regimes. We propose Wavelet Fine-Tuning (WaveFT), a novel PEFT method that learns highly sparse updates in the wavelet domain of residual matrices. WaveFT allows precise control of trainable parameters, offering fine-grained capacity adjustment and excelling with remarkably low parameter count, potentially far fewer than LoRA's minimum, ideal for extreme parameter-efficient scenarios. Evaluated on personalized text-to-image generation using Stable Diffusion XL as baseline, WaveFT significantly outperforms LoRA and other PEFT methods, especially at low parameter counts; achieving superior subject fidelity, prompt alignment, and image diversity. |
| title | Exploring Sparsity for Parameter Efficient Fine Tuning Using Wavelets |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Image and Video Processing Signal Processing |
| url | https://arxiv.org/abs/2505.12532 |