Cover Image

Stabilized Fine-Tuning with LoRA in Federated Learning: Mitigating the Side Effect of Client Size and Rank via the Scaling Factor

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Huang, Jiayu, Wu, Xiaohu, He, Tiantian, Lao, Qicheng
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MLAE: Masked LoRA Experts for Visual Parameter-Efficient Fine-Tuning
by: Wang, Junjie, et al.
Published: (2024)

LoRA-FAIR: Federated LoRA Fine-Tuning with Aggregation and Initialization Refinement
by: Bian, Jieming, et al.
Published: (2024)

DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models
by: Deng, Guanzhi, et al.
Published: (2026)

Rethinking the Rank Threshold for LoRA Fine-Tuning
by: Park, Juneyoung
Published: (2026)

Fed-pilot: Optimizing LoRA Allocation for Efficient Federated Fine-Tuning with Heterogeneous Clients
by: Zhang, Zikai, et al.
Published: (2024)

Stabilizing Decentralized Federated Fine-Tuning via Topology-Aware Alternating LoRA
by: Wang, Xiaoyu, et al.
Published: (2026)

CE-LoRA: Computation-Efficient LoRA Fine-Tuning for Language Models
by: Chen, Guanduo, et al.
Published: (2025)

ADF-LoRA: Alternating Low-Rank Aggregation for Decentralized Federated Fine-Tuning
by: Wang, Xiaoyu, et al.
Published: (2025)

L1RA: Dynamic Rank Assignment in LoRA Fine-Tuning
by: Singh, Raul, et al.
Published: (2025)

FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
by: Zhang, Haoran, et al.
Published: (2026)

LoRA$^2$ : Multi-Scale Low-Rank Approximations for Fine-Tuning Large Language Models
by: Zhang, Jia-Chen, et al.
Published: (2024)

Stable-LoRA: Stabilizing Feature Learning of Low-Rank Adaptation
by: Wu, Yize, et al.
Published: (2026)

SDFLoRA: Selective Decoupled Federated LoRA for Privacy-preserving Fine-tuning with Heterogeneous Clients
by: Shen, Zhikang, et al.
Published: (2026)

SHE-LoRA: Selective Homomorphic Encryption for Federated Tuning with Heterogeneous LoRA
by: Liu, Jianmin, et al.
Published: (2025)

FedMomentum: Preserving LoRA Training Momentum in Federated Fine-Tuning
by: Yan, Peishen, et al.
Published: (2026)

Adaptive LoRA Experts Allocation and Selection for Federated Fine-Tuning
by: Wang, Lei, et al.
Published: (2025)

LoRA-Edit: Controllable First-Frame-Guided Video Editing via Mask-Aware LoRA Fine-Tuning
by: Gao, Chenjian, et al.
Published: (2025)

FedTreeLoRA: Reconciling Statistical and Functional Heterogeneity in Federated LoRA Fine-Tuning
by: Bian, Jieming, et al.
Published: (2026)

LoRA-PAR: A Flexible Dual-System LoRA Partitioning Approach to Efficient LLM Fine-Tuning
by: Huang, Yining, et al.
Published: (2025)

HydraLoRA: An Asymmetric LoRA Architecture for Efficient Fine-Tuning
by: Tian, Chunlin, et al.
Published: (2024)

FDLoRA: Personalized Federated Learning of Large Language Model via Dual LoRA Tuning
by: QI, Jiaxing, et al.
Published: (2024)

Personalized Federated Fine-tuning for Heterogeneous Data: An Automatic Rank Learning Approach via Two-Level LoRA
by: Hao, Jie, et al.
Published: (2025)

Mitigating Unintended Memorization with LoRA in Federated Learning for LLMs
by: Bossy, Thierry, et al.
Published: (2025)

LoRAFusion: Efficient LoRA Fine-Tuning for LLMs
by: Zhu, Zhanda, et al.
Published: (2025)

KD-LoRA: A Hybrid Approach to Efficient Fine-Tuning with LoRA and Knowledge Distillation
by: Azimi, Rambod, et al.
Published: (2024)

Computational Limits of Low-Rank Adaptation (LoRA) Fine-Tuning for Transformer Models
by: Hu, Jerry Yao-Chieh, et al.
Published: (2024)

Localized LoRA: A Structured Low-Rank Approximation for Efficient Fine-Tuning
by: Barazandeh, Babak, et al.
Published: (2025)

GeLoRA: Geometric Adaptive Ranks For Efficient LoRA Fine-tuning
by: Ed-dib, Abdessalam, et al.
Published: (2024)

pFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuning
by: Yi, Liping, et al.
Published: (2023)

Bi-LoRA: Efficient Sharpness-Aware Minimization for Fine-Tuning Large-Scale Models
by: Liu, Yuhang, et al.
Published: (2025)

Wireless Federated Multi-Task LLM Fine-Tuning via Sparse-and-Orthogonal LoRA
by: Yang, Nuocheng, et al.
Published: (2026)

SC-LoRA: Balancing Efficient Fine-tuning and Knowledge Preservation via Subspace-Constrained LoRA
by: Luo, Minrui, et al.
Published: (2025)

LoRA-Edge: Tensor-Train-Assisted LoRA for Practical CNN Fine-Tuning on Edge Devices
by: Kwak, Hyunseok, et al.
Published: (2025)

FedLoRA-Optimizer: Federated LoRA Fine-Tuning with Global and Local Optimization in Heterogeneous Data Scenarios
by: Zhao, Jianzhe, et al.
Published: (2025)

FM-LoRA: Factorized Low-Rank Meta-Prompting for Continual Learning
by: Yu, Xiaobing, et al.
Published: (2025)

Bayesian-LoRA: LoRA based Parameter Efficient Fine-Tuning using Optimal Quantization levels and Rank Values trough Differentiable Bayesian Gates
by: Meo, Cristian, et al.
Published: (2024)

Two Stage Wireless Federated LoRA Fine-Tuning with Sparsified Orthogonal Updates
by: Kim, Bumjun, et al.
Published: (2025)

Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training
by: Zhang, Chengqian, et al.
Published: (2026)

LUNE: Efficient LLM Unlearning via LoRA Fine-Tuning with Negative Examples
by: Liu, Yezi, et al.
Published: (2025)

ALLoRA: Adaptive Learning Rate Mitigates LoRA Fatal Flaws
by: Huang, Hai, et al.
Published: (2024)