Federated Low-Rank Adaptation for Foundation Models: A Survey
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911228772745216 |
|---|---|
| author | Yang, Yiyuan Long, Guodong Lu, Qinghua Zhu, Liming Jiang, Jing Zhang, Chengqi |
| author_facet | Yang, Yiyuan Long, Guodong Lu, Qinghua Zhu, Liming Jiang, Jing Zhang, Chengqi |
| contents | Effectively leveraging private datasets remains a significant challenge in developing foundation models. Federated Learning (FL) has recently emerged as a collaborative framework that enables multiple users to fine-tune these models while mitigating data privacy risks. Meanwhile, Low-Rank Adaptation (LoRA) offers a resource-efficient alternative for fine-tuning foundation models by dramatically reducing the number of trainable parameters. This survey examines how LoRA has been integrated into federated fine-tuning for foundation models, an area we term FedLoRA, by focusing on three key challenges: distributed learning, heterogeneity, and efficiency. We further categorize existing work based on the specific methods used to address each challenge. Finally, we discuss open research questions and highlight promising directions for future investigation, outlining the next steps for advancing FedLoRA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_13502 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Federated Low-Rank Adaptation for Foundation Models: A Survey Yang, Yiyuan Long, Guodong Lu, Qinghua Zhu, Liming Jiang, Jing Zhang, Chengqi Machine Learning Effectively leveraging private datasets remains a significant challenge in developing foundation models. Federated Learning (FL) has recently emerged as a collaborative framework that enables multiple users to fine-tune these models while mitigating data privacy risks. Meanwhile, Low-Rank Adaptation (LoRA) offers a resource-efficient alternative for fine-tuning foundation models by dramatically reducing the number of trainable parameters. This survey examines how LoRA has been integrated into federated fine-tuning for foundation models, an area we term FedLoRA, by focusing on three key challenges: distributed learning, heterogeneity, and efficiency. We further categorize existing work based on the specific methods used to address each challenge. Finally, we discuss open research questions and highlight promising directions for future investigation, outlining the next steps for advancing FedLoRA. |
| title | Federated Low-Rank Adaptation for Foundation Models: A Survey |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2505.13502 |