Cover Image

FLoE: Fisher-Based Layer Selection for Efficient Sparse Adaptation of Low-Rank Experts

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Xinyi, Gao, Lirong, Wang, Haobo, Zhang, Yiming, Zhao, Junbo
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

zFLoRA: Zero-Latency Fused Low-Rank Adapters
by: Gowda, Dhananjaya, et al.
Published: (2025)

DropLoRA: Sparse Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
by: Zhang, Haojie
Published: (2025)

XMoE: Sparse Models with Fine-grained and Adaptive Expert Selection
by: Yang, Yuanhang, et al.
Published: (2024)

ResidualTransformer: Residual Low-Rank Learning with Weight-Sharing for Transformer Layers
by: Wang, Yiming, et al.
Published: (2023)

Ensembles of Low-Rank Expert Adapters
by: Li, Yinghao, et al.
Published: (2025)

Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptation
by: Li, Shiwei, et al.
Published: (2025)

OMoE: Diversifying Mixture of Low-Rank Adaptation by Orthogonal Finetuning
by: Feng, Jinyuan, et al.
Published: (2025)

FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations
by: Wang, Ziyao, et al.
Published: (2024)

FLoRA: Sample-Efficient Preference-based RL via Low-Rank Style Adaptation of Reward Functions
by: Marta, Daniel, et al.
Published: (2025)

Towards Understanding the Nature of Attention with Low-Rank Sparse Decomposition
by: He, Zhengfu, et al.
Published: (2025)

LoRA-GA: Low-Rank Adaptation with Gradient Approximation
by: Wang, Shaowen, et al.
Published: (2024)

MAP: Revisiting Weight Decomposition for Low-Rank Adaptation
by: Si, Chongjie, et al.
Published: (2025)

From Parameters to Data: A Task-Parameter-Guided Fine-Tuning Pipeline for Efficient LLM Alignment
by: Chen, Hao, et al.
Published: (2026)

HELLoRA: Hot Experts Layer-Level Low-Rank Adaptation for Mixture-of-Experts Models
by: Wei, Jia, et al.
Published: (2026)

Robust and Efficient Fine-tuning of LLMs with Bayesian Reparameterization of Low-Rank Adaptation
by: Sengupta, Ayan, et al.
Published: (2024)

MoR: Mixture of Ranks for Low-Rank Adaptation Tuning
by: Tang, Chuanyu, et al.
Published: (2024)

LoRA+: Efficient Low Rank Adaptation of Large Models
by: Hayou, Soufiane, et al.
Published: (2024)

Diversifying the Expert Knowledge for Task-Agnostic Pruning in Sparse Mixture-of-Experts
by: Zhang, Zeliang, et al.
Published: (2024)

BiDoRA: Bi-level Optimization-Based Weight-Decomposed Low-Rank Adaptation
by: Qin, Peijia, et al.
Published: (2024)

MALoRA: Mixture of Asymmetric Low-Rank Adaptation for Enhanced Multi-Task Learning
by: Wang, Xujia, et al.
Published: (2024)

Prompt Candidates, then Distill: A Teacher-Student Framework for LLM-driven Data Annotation
by: Xia, Mingxuan, et al.
Published: (2025)

RotMoLE: Enhancing Mixture of Low-Rank Experts through Rotational Gating Mechanism
by: Sun, Mengyang, et al.
Published: (2026)

AutoLoRA: Automatically Tuning Matrix Ranks in Low-Rank Adaptation Based on Meta Learning
by: Zhang, Ruiyi, et al.
Published: (2024)

SwitchLoRA: Switched Low-Rank Adaptation Can Learn Full-Rank Information
by: Zhou, Kaiye, et al.
Published: (2024)

LSR-Adapt: Ultra-Efficient Parameter Tuning with Matrix Low Separation Rank Kernel Adaptation
by: Li, Xin, et al.
Published: (2025)

Contextually Guided Transformers via Low-Rank Adaptation
by: Zhmoginov, Andrey, et al.
Published: (2025)

A Bayesian Interpretation of Adaptive Low-Rank Adaptation
by: Chen, Haolin, et al.
Published: (2024)

SBoRA: Low-Rank Adaptation with Regional Weight Updates
by: Po, Lai-Man, et al.
Published: (2024)

Unlocking the Global Synergies in Low-Rank Adapters
by: Zhang, Zixi, et al.
Published: (2024)

The Expressive Power of Low-Rank Adaptation
by: Zeng, Yuchen, et al.
Published: (2023)

SARA: Singular-Value Based Adaptive Low-Rank Adaption
by: Gu, Jihao, et al.
Published: (2024)

LoLA: Low-Rank Linear Attention With Sparse Caching
by: McDermott, Luke, et al.
Published: (2025)

BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models
by: Wang, Yibin, et al.
Published: (2024)

QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning
by: Rajabzadeh, Hossein, et al.
Published: (2024)

ALoRE: Efficient Visual Adaptation via Aggregating Low Rank Experts
by: Du, Sinan, et al.
Published: (2024)

CSULoRA: Closest Safe Update Low-Rank Adaptation
by: Breneur, Oleksandr Marchenko, et al.
Published: (2026)

MTL-LoRA: Low-Rank Adaptation for Multi-Task Learning
by: Yang, Yaming, et al.
Published: (2024)

Batched Low-Rank Adaptation of Foundation Models
by: Wen, Yeming, et al.
Published: (2023)

Variational Low-Rank Adaptation Using IVON
by: Cong, Bai, et al.
Published: (2024)

SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts
by: Muzio, Alexandre, et al.
Published: (2024)