Cover Image

Mitigating Catastrophic Forgetting with Adaptive Transformer Block Expansion in Federated Fine-Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Huo, Yujia, Liu, Jianchun, Xu, Hongli, Ma, Zhenguo, Wang, Shilong, Huang, Liusheng
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

FedQuad: Adaptive Layer-wise LoRA Deployment and Activation Quantization for Federated Fine-Tuning
by: Li, Rukuo, et al.
Published: (2025)

Towards Communication-Efficient Decentralized Federated Graph Learning over Non-IID Data
by: Wang, Shilong, et al.
Published: (2025)

Efficient Federated Fine-Tuning of Large Language Models with Layer Dropout
by: Wang, Shilong, et al.
Published: (2025)

Caesar: A Low-deviation Compression Approach for Efficient Federated Learning
by: Yan, Jiaming, et al.
Published: (2024)

Many Hands Make Light Work: Accelerating Edge Inference via Multi-Client Collaborative Caching
by: Liang, Wenyi, et al.
Published: (2024)

Collaborative Speculative Inference for Efficient LLM Inference Serving
by: Gao, Luyao, et al.
Published: (2025)

Accelerating End-Cloud Collaborative Inference via Near Bubble-free Pipeline Optimization
by: Gao, Luyao, et al.
Published: (2024)

A Robust Federated Learning Framework for Undependable Devices at Scale
by: Wang, Shilong, et al.
Published: (2024)

SemiSFL: Split Federated Learning on Unlabeled and Non-IID Data
by: Xu, Yang, et al.
Published: (2023)

FedRef: Bayesian Fine-Tuning using a Reference Model to Mitigate Catastrophic Forgetting for Heterogeneous Federated Learning
by: Yoon, Taehwan, et al.
Published: (2025)

Federated Learning Under Temporal Drift -- Mitigating Catastrophic Forgetting via Experience Replay
by: Kokkula, Sahasra, et al.
Published: (2026)

Adaptive Parameter-Efficient Federated Fine-Tuning on Heterogeneous Devices
by: Liu, Jun, et al.
Published: (2024)

Catastrophic Forgetting Resilient One-Shot Incremental Federated Learning
by: Zaland, Obaidullah, et al.
Published: (2026)

ParallelSFL: A Novel Split Federated Learning Framework Tackling Heterogeneity Issues
by: Liao, Yunming, et al.
Published: (2024)

Bandwidth-Aware and Cost-Efficient Pipeline Parallel Scheduling in Geo-Distributed LLM Training
by: Zhang, Han, et al.
Published: (2026)

ParaBlock: Communication-Computation Parallel Block Coordinate Federated Learning for Large Language Models
by: Wang, Yujia, et al.
Published: (2025)

SplitFT: An Adaptive Federated Split Learning System For LLMs Fine-Tuning
by: Shan, Yimeng, et al.
Published: (2026)

Air-FedGA: A Grouping Asynchronous Federated Learning Mechanism Exploiting Over-the-air Computation
by: Ma, Qianpiao, et al.
Published: (2025)

Cross-region Model Training with Communication-Computation Overlapping and Delay Compensation
by: Zhu, Ying, et al.
Published: (2025)

Cyclical Weight Consolidation: Towards Solving Catastrophic Forgetting in Serial Federated Learning
by: Song, Haoyue, et al.
Published: (2024)

Communication-Efficient Federated Fine-Tuning
by: Theologitis, Michael, et al.
Published: (2025)

Fed-pilot: Optimizing LoRA Allocation for Efficient Federated Fine-Tuning with Heterogeneous Clients
by: Zhang, Zikai, et al.
Published: (2024)

ToFU: Transforming How Federated Learning Systems Forget User Data
by: Tran, Van-Tuan, et al.
Published: (2025)

Fed-HeLLo: Efficient Federated Foundation Model Fine-Tuning with Heterogeneous LoRA Allocation
by: Zhang, Zikai, et al.
Published: (2025)

Exploring Selective Layer Fine-Tuning in Federated Learning
by: Sun, Yuchang, et al.
Published: (2024)

Stragglers Can Contribute More: Uncertainty-Aware Distillation for Asynchronous Federated Learning
by: Wang, Yujia, et al.
Published: (2025)

Revisiting Federated Fine-Tuning: A Single Communication Round is Enough for Foundation Models
by: Wang, Ziyao, et al.
Published: (2024)

Stabilizing Decentralized Federated Fine-Tuning via Topology-Aware Alternating LoRA
by: Wang, Xiaoyu, et al.
Published: (2026)

Heterogeneous Federated Fine-Tuning with Parallel One-Rank Adaptation
by: Zhang, Zikai, et al.
Published: (2026)

Flexible Personalized Split Federated Learning for On-Device Fine-Tuning of Foundation Models
by: Yuan, Tianjun, et al.
Published: (2025)

FUPareto: Bridging the Forgetting-Utility Gap in Federated Unlearning via Pareto Augmented Optimization
by: Wang, Zeyan, et al.
Published: (2026)

HLoRA: Efficient Federated Learning System for LLM Heterogeneous Fine-Tuning
by: Liu, Qianli, et al.
Published: (2025)

FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations
by: Wang, Ziyao, et al.
Published: (2024)

Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Models
by: Cho, Yae Jee, et al.
Published: (2024)

Deadline-Aware Online Scheduling for LLM Fine-Tuning with Spot Market Predictions
by: Kong, Linggao, et al.
Published: (2025)

EcoLoRA: Communication-Efficient Federated Fine-Tuning of Large Language Models
by: Liu, Han, et al.
Published: (2025)

A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning
by: Lyu, Chenghao, et al.
Published: (2024)

FedCAda: Adaptive Client-Side Optimization for Accelerated and Stable Federated Learning
by: Zhou, Liuzhi, et al.
Published: (2024)

Adaptive Federated Learning with Auto-Tuned Clients
by: Kim, Junhyung Lyle, et al.
Published: (2023)

CLLoRA: An Approach to Measure the Effects of the Context Length for LLM Fine-Tuning
by: Zhang, Ping, et al.
Published: (2025)