Copertina

FLoE: Fisher-Based Layer Selection for Efficient Sparse Adaptation of Low-Rank Experts

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wang, Xinyi, Gao, Lirong, Wang, Haobo, Zhang, Yiming, Zhao, Junbo
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

zFLoRA: Zero-Latency Fused Low-Rank Adapters
di: Gowda, Dhananjaya, et al.
Pubblicazione: (2025)

DropLoRA: Sparse Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
di: Zhang, Haojie
Pubblicazione: (2025)

XMoE: Sparse Models with Fine-grained and Adaptive Expert Selection
di: Yang, Yuanhang, et al.
Pubblicazione: (2024)

ResidualTransformer: Residual Low-Rank Learning with Weight-Sharing for Transformer Layers
di: Wang, Yiming, et al.
Pubblicazione: (2023)

Ensembles of Low-Rank Expert Adapters
di: Li, Yinghao, et al.
Pubblicazione: (2025)

Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptation
di: Li, Shiwei, et al.
Pubblicazione: (2025)

OMoE: Diversifying Mixture of Low-Rank Adaptation by Orthogonal Finetuning
di: Feng, Jinyuan, et al.
Pubblicazione: (2025)

FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations
di: Wang, Ziyao, et al.
Pubblicazione: (2024)

FLoRA: Sample-Efficient Preference-based RL via Low-Rank Style Adaptation of Reward Functions
di: Marta, Daniel, et al.
Pubblicazione: (2025)

Towards Understanding the Nature of Attention with Low-Rank Sparse Decomposition
di: He, Zhengfu, et al.
Pubblicazione: (2025)

LoRA-GA: Low-Rank Adaptation with Gradient Approximation
di: Wang, Shaowen, et al.
Pubblicazione: (2024)

MAP: Revisiting Weight Decomposition for Low-Rank Adaptation
di: Si, Chongjie, et al.
Pubblicazione: (2025)

From Parameters to Data: A Task-Parameter-Guided Fine-Tuning Pipeline for Efficient LLM Alignment
di: Chen, Hao, et al.
Pubblicazione: (2026)

HELLoRA: Hot Experts Layer-Level Low-Rank Adaptation for Mixture-of-Experts Models
di: Wei, Jia, et al.
Pubblicazione: (2026)

Robust and Efficient Fine-tuning of LLMs with Bayesian Reparameterization of Low-Rank Adaptation
di: Sengupta, Ayan, et al.
Pubblicazione: (2024)

MoR: Mixture of Ranks for Low-Rank Adaptation Tuning
di: Tang, Chuanyu, et al.
Pubblicazione: (2024)

LoRA+: Efficient Low Rank Adaptation of Large Models
di: Hayou, Soufiane, et al.
Pubblicazione: (2024)

Diversifying the Expert Knowledge for Task-Agnostic Pruning in Sparse Mixture-of-Experts
di: Zhang, Zeliang, et al.
Pubblicazione: (2024)

BiDoRA: Bi-level Optimization-Based Weight-Decomposed Low-Rank Adaptation
di: Qin, Peijia, et al.
Pubblicazione: (2024)

MALoRA: Mixture of Asymmetric Low-Rank Adaptation for Enhanced Multi-Task Learning
di: Wang, Xujia, et al.
Pubblicazione: (2024)

Prompt Candidates, then Distill: A Teacher-Student Framework for LLM-driven Data Annotation
di: Xia, Mingxuan, et al.
Pubblicazione: (2025)

RotMoLE: Enhancing Mixture of Low-Rank Experts through Rotational Gating Mechanism
di: Sun, Mengyang, et al.
Pubblicazione: (2026)

AutoLoRA: Automatically Tuning Matrix Ranks in Low-Rank Adaptation Based on Meta Learning
di: Zhang, Ruiyi, et al.
Pubblicazione: (2024)

SwitchLoRA: Switched Low-Rank Adaptation Can Learn Full-Rank Information
di: Zhou, Kaiye, et al.
Pubblicazione: (2024)

LSR-Adapt: Ultra-Efficient Parameter Tuning with Matrix Low Separation Rank Kernel Adaptation
di: Li, Xin, et al.
Pubblicazione: (2025)

Contextually Guided Transformers via Low-Rank Adaptation
di: Zhmoginov, Andrey, et al.
Pubblicazione: (2025)

A Bayesian Interpretation of Adaptive Low-Rank Adaptation
di: Chen, Haolin, et al.
Pubblicazione: (2024)

SBoRA: Low-Rank Adaptation with Regional Weight Updates
di: Po, Lai-Man, et al.
Pubblicazione: (2024)

Unlocking the Global Synergies in Low-Rank Adapters
di: Zhang, Zixi, et al.
Pubblicazione: (2024)

The Expressive Power of Low-Rank Adaptation
di: Zeng, Yuchen, et al.
Pubblicazione: (2023)

SARA: Singular-Value Based Adaptive Low-Rank Adaption
di: Gu, Jihao, et al.
Pubblicazione: (2024)

LoLA: Low-Rank Linear Attention With Sparse Caching
di: McDermott, Luke, et al.
Pubblicazione: (2025)

BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models
di: Wang, Yibin, et al.
Pubblicazione: (2024)

QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning
di: Rajabzadeh, Hossein, et al.
Pubblicazione: (2024)

ALoRE: Efficient Visual Adaptation via Aggregating Low Rank Experts
di: Du, Sinan, et al.
Pubblicazione: (2024)

CSULoRA: Closest Safe Update Low-Rank Adaptation
di: Breneur, Oleksandr Marchenko, et al.
Pubblicazione: (2026)

MTL-LoRA: Low-Rank Adaptation for Multi-Task Learning
di: Yang, Yaming, et al.
Pubblicazione: (2024)

Batched Low-Rank Adaptation of Foundation Models
di: Wen, Yeming, et al.
Pubblicazione: (2023)

Variational Low-Rank Adaptation Using IVON
di: Cong, Bai, et al.
Pubblicazione: (2024)

SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts
di: Muzio, Alexandre, et al.
Pubblicazione: (2024)