Cover Image

Efficient Federated Fine-Tuning of Large Language Models with Layer Dropout

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Shilong, Liu, Jianchun, Xu, Hongli, Yan, Jiaming, Gao, Xianjun
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Mitigating Catastrophic Forgetting with Adaptive Transformer Block Expansion in Federated Fine-Tuning
by: Huo, Yujia, et al.
Published: (2025)

Adaptive Parameter-Efficient Federated Fine-Tuning on Heterogeneous Devices
by: Liu, Jun, et al.
Published: (2024)

FLoRIST: Singular Value Thresholding for Efficient and Accurate Federated Fine-Tuning of Large Language Models
by: Ramesh, Hariharan, et al.
Published: (2025)

Caesar: A Low-deviation Compression Approach for Efficient Federated Learning
by: Yan, Jiaming, et al.
Published: (2024)

Efficient Federated Learning with Heterogeneous Data and Adaptive Dropout
by: Liu, Ji, et al.
Published: (2025)

HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
by: Lin, Zheng, et al.
Published: (2025)

FedQuad: Adaptive Layer-wise LoRA Deployment and Activation Quantization for Federated Fine-Tuning
by: Li, Rukuo, et al.
Published: (2025)

SFPrompt: Communication-Efficient Split Federated Fine-Tuning for Large Pre-Trained Models over Resource-Limited Devices
by: Cao, Linxiao, et al.
Published: (2024)

A Robust Federated Learning Framework for Undependable Devices at Scale
by: Wang, Shilong, et al.
Published: (2024)

DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Model
by: Gao, Chao, et al.
Published: (2024)

Enhancing Data Quality in Federated Fine-Tuning of Foundation Models
by: Zhao, Wanru, et al.
Published: (2024)

Efficient Deployment of Large Language Models on Resource-constrained Devices
by: Yao, Zhiwei, et al.
Published: (2025)

Learning Like Humans: Resource-Efficient Federated Fine-Tuning through Cognitive Developmental Stages
by: Wu, Yebo, et al.
Published: (2025)

The Future of Large Language Model Pre-training is Federated
by: Sani, Lorenzo, et al.
Published: (2024)

Mitigating Persistent Client Dropout in Asynchronous Decentralized Federated Learning
by: Stępka, Ignacy, et al.
Published: (2025)

A Closer Look at Personalized Fine-Tuning in Heterogeneous Federated Learning
by: Chen, Minghui, et al.
Published: (2025)

Adaptive Rank Allocation for Federated Parameter-Efficient Fine-Tuning of Language Models
by: Wu, Fei, et al.
Published: (2025)

Collaborative Speculative Inference for Efficient LLM Inference Serving
by: Gao, Luyao, et al.
Published: (2025)

LoRAFusion: Efficient LoRA Fine-Tuning for LLMs
by: Zhu, Zhanda, et al.
Published: (2025)

CG-FedLLM: How to Compress Gradients in Federated Fune-tuning for Large Language Models
by: Wu, Huiwen, et al.
Published: (2024)

FedRef: Bayesian Fine-Tuning using a Reference Model to Mitigate Catastrophic Forgetting for Heterogeneous Federated Learning
by: Yoon, Taehwan, et al.
Published: (2025)

Resource-Efficient Personal Large Language Models Fine-Tuning with Collaborative Edge Computing
by: Ye, Shengyuan, et al.
Published: (2024)

SPD-CFL: Stepwise Parameter Dropout for Efficient Continual Federated Learning
by: Yang, Yuning, et al.
Published: (2024)

Towards the Next Frontier of LLMs, Training on Private Data: A Cross-Domain Benchmark for Federated Fine-Tuning
by: Jimenez-Gutierrez, Daniel M., et al.
Published: (2026)

Towards Communication-Efficient Decentralized Federated Graph Learning over Non-IID Data
by: Wang, Shilong, et al.
Published: (2025)

Federated Fine-Tuning of Sparsely-Activated Large Language Models on Resource-Constrained Devices
by: Chen, Fahao, et al.
Published: (2025)

Symbiosis: Multi-Adapter Inference and Fine-Tuning
by: Gupta, Saransh, et al.
Published: (2025)

HAFLQ: Heterogeneous Adaptive Federated LoRA Fine-tuned LLM with Quantization
by: Su, Yang, et al.
Published: (2024)

Exploring Selective Layer Fine-Tuning in Federated Learning
by: Sun, Yuchang, et al.
Published: (2024)

Communication-Efficient Federated Fine-Tuning
by: Theologitis, Michael, et al.
Published: (2025)

FedTrans: Efficient Federated Learning via Multi-Model Transformation
by: Zhu, Yuxuan, et al.
Published: (2024)

Fine-Tuning GPT-5 for GPU Kernel Generation
by: Tehrani, Ali, et al.
Published: (2026)

Fisher Information-based Efficient Curriculum Federated Learning with Large Language Models
by: Liu, Ji, et al.
Published: (2024)

FedPop: Federated Population-based Hyperparameter Tuning
by: Chen, Haokun, et al.
Published: (2023)

Fed-HeLLo: Efficient Federated Foundation Model Fine-Tuning with Heterogeneous LoRA Allocation
by: Zhang, Zikai, et al.
Published: (2025)

SPD: Sync-Point Drop for Efficient Tensor Parallelism of Large Language Models
by: Kim, Han-Byul, et al.
Published: (2025)

FedTLU: Federated Learning with Targeted Layer Updates
by: Park, Jong-Ik, et al.
Published: (2024)

Federated Neural Architecture Search with Model-Agnostic Meta Learning
by: Huang, Xinyuan, et al.
Published: (2025)

Online Client Scheduling and Resource Allocation for Efficient Federated Edge Learning
by: Gao, Zhidong, et al.
Published: (2024)

Efficient Asynchronous Federated Learning with Sparsification and Quantization
by: Jia, Juncheng, et al.
Published: (2023)