Cover Image

Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Raje, Arian, Askin, Baris, Jhunjhunwala, Divyansh, Joshi, Gauri
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MELINOE: Fine-Tuning Enables Memory-Efficient Inference for Mixture-of-Experts Models
by: Raje, Arian, et al.
Published: (2026)

FedFisher: Leveraging Fisher Information for One-Shot Federated Learning
by: Jhunjhunwala, Divyansh, et al.
Published: (2024)

Internal Planning in Language Models: Characterizing Horizon and Branch Awareness
by: Ustaomeroglu, Muhammed, et al.
Published: (2025)

Initialization Matters: Unraveling the Impact of Pre-Training on Federated Learning
by: Jhunjhunwala, Divyansh, et al.
Published: (2025)

Emergent and Subliminal Misalignment Through the Lens of Data-Mediated Transfer
by: Askin, Baris, et al.
Published: (2026)

Erasure Coded Neural Network Inference via Fisher Averaging
by: Jhunjhunwala, Divyansh, et al.
Published: (2024)

Polynomial Expansion Rank Adaptation: Enhancing Low-Rank Fine-Tuning with High-Order Interactions
by: Zhang, Wenhao, et al.
Published: (2026)

Federated Communication-Efficient Multi-Objective Optimization
by: Askin, Baris, et al.
Published: (2024)

FedRPCA: Enhancing Federated LoRA Aggregation Using Robust PCA
by: Jhunjhunwala, Divyansh, et al.
Published: (2025)

GraLoRA: Granular Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
by: Jung, Yeonjoon, et al.
Published: (2025)

Signs Beat Floats: Low-Rank Double-Binary Adaptation for On-Device Fine-Tuning
by: Fujisawa, Yoshihiko, et al.
Published: (2026)

Enhancing Federated Domain Adaptation with Multi-Domain Prototype-Based Federated Fine-Tuning
by: Zhang, Jingyuan, et al.
Published: (2024)

Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training
by: Zhang, Chengqian, et al.
Published: (2026)

FedAST: Federated Asynchronous Simultaneous Training
by: Askin, Baris, et al.
Published: (2024)

LOCUS: Low-Dimensional Model Embeddings for Efficient Model Exploration, Comparison, and Selection
by: Patel, Shivam, et al.
Published: (2026)

Computational Limits of Low-Rank Adaptation (LoRA) Fine-Tuning for Transformer Models
by: Hu, Jerry Yao-Chieh, et al.
Published: (2024)

Aggregating Low Rank Adapters in Federated Fine-tuning
by: Trautmann, Evelyn, et al.
Published: (2025)

MoR: Mixture of Ranks for Low-Rank Adaptation Tuning
by: Tang, Chuanyu, et al.
Published: (2024)

RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptation
by: Liu, Jun, et al.
Published: (2025)

FL-TAC: Enhanced Fine-Tuning in Federated Learning via Low-Rank, Task-Specific Adapter Clustering
by: Ping, Siqi, et al.
Published: (2024)

Not All Turns Are Equally Hard: Adaptive Thinking Budgets For Efficient Multi-Turn Reasoning
by: Jali, Neharika, et al.
Published: (2026)

Memory-Efficient Fine-Tuning via Low-Rank Activation Compression
by: Shi, Jiang-Xin, et al.
Published: (2025)

PubSwap: Public-Data Off-Policy Coordination for Federated RLVR
by: Nayak, Anupam, et al.
Published: (2026)

AFLoRA: Adaptive Freezing of Low Rank Adaptation in Parameter Efficient Fine-Tuning of Large Models
by: Liu, Zeyu, et al.
Published: (2024)

A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models
by: Sun, Mengyang, et al.
Published: (2025)

LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models
by: Yang, Yifan, et al.
Published: (2024)

Prompt Tuning Strikes Back: Customizing Foundation Models with Low-Rank Prompt Adaptation
by: Jain, Abhinav, et al.
Published: (2024)

Sparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models
by: Zhang, Longteng, et al.
Published: (2026)

GaLore$+$: Boosting Low-Rank Adaptation for LLMs with Cross-Head Projection
by: Liao, Xutao, et al.
Published: (2024)

FedHL: Federated Learning for Heterogeneous Low-Rank Adaptation via Unbiased Aggregation
by: Peng, Zihao, et al.
Published: (2025)

Towards Unified Multi-task EEG Analysis with Low-Rank Adaptation
by: Dai, Sicheng, et al.
Published: (2026)

The Primacy of Magnitude in Low-Rank Adaptation
by: Zhang, Zicheng, et al.
Published: (2025)

Mixture-of-Subspaces in Low-Rank Adaptation
by: Wu, Taiqiang, et al.
Published: (2024)

Gradual Fine-Tuning with Graph Routing for Multi-Source Unsupervised Domain Adaptation
by: Ma, Yao, et al.
Published: (2024)

Fine-Tuned In-Context Learners for Efficient Adaptation
by: Bornschein, Jorg, et al.
Published: (2025)

Preventing Rank Collapse in Federated Low-Rank Adaptation with Client Heterogeneity
by: Wu, Fei, et al.
Published: (2026)

Rethinking the Rank Threshold for LoRA Fine-Tuning
by: Park, Juneyoung
Published: (2026)

ThanoRA: Task Heterogeneity-Aware Multi-Task Low-Rank Adaptation
by: Liang, Jian, et al.
Published: (2025)

ARD-LoRA: Dynamic Rank Allocation for Parameter-Efficient Fine-Tuning of Foundation Models with Heterogeneous Adaptation Needs
by: Shinwari, Haseeb Ullah Khan, et al.
Published: (2025)

AutoLoRA: Automatically Tuning Matrix Ranks in Low-Rank Adaptation Based on Meta Learning
by: Zhang, Ruiyi, et al.
Published: (2024)