Cover Image

Towards Federated Low-Rank Adaptation of Language Models with Rank Heterogeneity

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Byun, Yuji, Lee, Jaeho
Format:	Preprint
Published:	2024
Subjects:	Distributed, Parallel, and Cluster Computing Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations
by: Wang, Ziyao, et al.
Published: (2024)

Preventing Rank Collapse in Federated Low-Rank Adaptation with Client Heterogeneity
by: Wu, Fei, et al.
Published: (2026)

Towards Robust and Efficient Federated Low-Rank Adaptation with Heterogeneous Clients
by: Koo, Jabin, et al.
Published: (2024)

The Panaceas for Improving Low-Rank Decomposition in Communication-Efficient Federated Learning
by: Li, Shiwei, et al.
Published: (2025)

BOOST: BOttleneck-Optimized Scalable Training Framework for Low-Rank Large Language Models
by: Wang, Zhengyang, et al.
Published: (2025)

Heterogeneous Federated Fine-Tuning with Parallel One-Rank Adaptation
by: Zhang, Zikai, et al.
Published: (2026)

RBLA: Rank-Based-LoRA-Aggregation for Fine-tuning Heterogeneous Models in FLaaS
by: Chen, Shuaijun, et al.
Published: (2024)

Ranking-based Client Selection with Imitation Learning for Efficient Federated Learning
by: Tian, Chunlin, et al.
Published: (2024)

Federated Aggregation of Mallows Rankings: A Comparative Analysis of Borda and Lehmer Coding
by: Sima, Jin, et al.
Published: (2024)

AutoRank: MCDA Based Rank Personalization for LoRA-Enabled Distributed Learning
by: Chen, Shuaijun, et al.
Published: (2024)

Communication-Efficient Personalized Federal Graph Learning via Low-Rank Decomposition
by: Liu, Ruyue, et al.
Published: (2024)

Heterogeneous Federated Learning with Prototype Alignment and Upscaling
by: Lee, Gyuejeong, et al.
Published: (2025)

DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptation
by: Xu, Jie, et al.
Published: (2024)

Federated Model Heterogeneous Matryoshka Representation Learning
by: Yi, Liping, et al.
Published: (2024)

Hypernetworks for Model-Heterogeneous Personalized Federated Learning
by: Zhang, Chen, et al.
Published: (2025)

Knowledge-Driven Federated Graph Learning on Model Heterogeneity
by: Wu, Zhengyu, et al.
Published: (2025)

Federated Low-Rank Tensor Estimation for Multimodal Image Reconstruction
by: Van Nguyen, Anh, et al.
Published: (2025)

Robust Model Aggregation for Heterogeneous Federated Learning: Analysis and Optimizations
by: Shao, Yumeng, et al.
Published: (2024)

Breaking the Memory Wall for Heterogeneous Federated Learning via Model Splitting
by: Tian, Chunlin, et al.
Published: (2024)

Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Models
by: Cho, Yae Jee, et al.
Published: (2024)

FedMOA: Federated GRPO for Personalized Reasoning LLMs under Heterogeneous Rewards
by: Wang, Ziyao, et al.
Published: (2026)

Low-Rank GEMM: Efficient Matrix Multiplication via Low-Rank Approximation with FP8 Acceleration
by: Metere, Alfredo
Published: (2025)

FLASH: Federated Learning Across Simultaneous Heterogeneities
by: Chang, Xiangyu, et al.
Published: (2024)

STHFL: Spatio-Temporal Heterogeneous Federated Learning
by: Guo, Shunxin, et al.
Published: (2025)

RankMap: Priority-Aware Multi-DNN Manager for Heterogeneous Embedded Devices
by: Karatzas, Andreas, et al.
Published: (2024)

Heterogeneous Federated Learning with Convolutional and Spiking Neural Networks
by: Yu, Yingchao, et al.
Published: (2024)

Heterogeneity: An Open Challenge for Federated On-board Machine Learning
by: Hartmann, Maria, et al.
Published: (2024)

Convergence Analysis of Split Federated Learning on Heterogeneous Data
by: Han, Pengchao, et al.
Published: (2024)

An Efficient Subspace Algorithm for Federated Learning on Heterogeneous Data
by: Zhang, Jiaojiao, et al.
Published: (2025)

Tackling Privacy Heterogeneity in Differentially Private Federated Learning
by: Xu, Ruichen, et al.
Published: (2026)

Heterogeneous Parallelism for Multimodal Large Language Model Training
by: Karnati, Yashaswi, et al.
Published: (2026)

Efficient Heterogeneous Large Language Model Decoding with Model-Attention Disaggregation
by: Chen, Shaoyuan, et al.
Published: (2024)

pFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuning
by: Yi, Liping, et al.
Published: (2023)

FedSSA: Semantic Similarity-based Aggregation for Efficient Model-Heterogeneous Personalized Federated Learning
by: Yi, Liping, et al.
Published: (2023)

Not All Edges are Equally Robust: Evaluating the Robustness of Ranking-Based Federated Learning
by: Gong, Zirui, et al.
Published: (2025)

Achieving Linear Speedup in Asynchronous Federated Learning with Heterogeneous Clients
by: Wang, Xiaolu, et al.
Published: (2024)

Non-Federated Multi-Task Split Learning for Heterogeneous Sources
by: Zheng, Yilin, et al.
Published: (2024)

Apodotiko: Enabling Efficient Serverless Federated Learning in Heterogeneous Environments
by: Chadha, Mohak, et al.
Published: (2024)

Addressing Skewed Heterogeneity via Federated Prototype Rectification with Personalization
by: Guo, Shunxin, et al.
Published: (2024)

On the Convergence Rates of Federated Q-Learning across Heterogeneous Environments
by: Wang, Leo Muxing, et al.
Published: (2024)