Cover Image

MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Jingfan, Zhao, Yi, Chen, Dan, Tian, Xing, Zheng, Huanran, Zhu, Wei
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ALoRA: Allocating Low-Rank Adaptation for Fine-tuning Large Language Models
by: Liu, Zequan, et al.
Published: (2024)

MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning
by: Wang, Hanqing, et al.
Published: (2024)

ShareLoRA: Parameter Efficient and Robust Large Language Model Fine-tuning via Shared Low-Rank Adaptation
by: Song, Yurun, et al.
Published: (2024)

LoRA-FA: Efficient and Effective Low Rank Representation Fine-tuning
by: Zhang, Longteng, et al.
Published: (2023)

LoRA+: Efficient Low Rank Adaptation of Large Models
by: Hayou, Soufiane, et al.
Published: (2024)

QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning
by: Rajabzadeh, Hossein, et al.
Published: (2024)

DenseLoRA: Dense Low-Rank Adaptation of Large Language Models
by: Mu, Lin, et al.
Published: (2025)

DropLoRA: Sparse Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
by: Zhang, Haojie
Published: (2025)

DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models
by: Deng, Guanzhi, et al.
Published: (2026)

LoRA$^2$ : Multi-Scale Low-Rank Approximations for Fine-Tuning Large Language Models
by: Zhang, Jia-Chen, et al.
Published: (2024)

BA-LoRA: Bias-Alleviating Low-Rank Adaptation to Mitigate Catastrophic Inheritance in Large Language Models
by: Chang, Yupeng, et al.
Published: (2024)

LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
by: Chen, Yukang, et al.
Published: (2023)

TsqLoRA: Towards Sensitivity and Quality Low-Rank Adaptation for Efficient Fine-Tuning
by: Chen, Yu, et al.
Published: (2025)

TriAdaptLoRA: Brain-Inspired Triangular Adaptive Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
by: Liang, Yao, et al.
Published: (2025)

NLoRA: Nyström-Initiated Low-Rank Adaptation for Large Language Models
by: Guo, Chenlu, et al.
Published: (2025)

ElaLoRA: Elastic & Learnable Low-Rank Adaptation for Efficient Model Fine-Tuning
by: Chang, Huandong, et al.
Published: (2025)

BeamLoRA: Beam-Constraint Low-Rank Adaptation
by: Gu, Naibin, et al.
Published: (2025)

Dynamic Adaptation of LoRA Fine-Tuning for Efficient and Task-Specific Optimization of Large Language Models
by: Liao, Xiaoxuan, et al.
Published: (2025)

LoRS: Efficient Low-Rank Adaptation for Sparse Large Language Model
by: Hu, Yuxuan, et al.
Published: (2025)

MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts
by: Li, Dengchun, et al.
Published: (2024)

RoseLoRA: Row and Column-wise Sparse Low-rank Adaptation of Pre-trained Language Model for Knowledge Editing and Fine-tuning
by: Wang, Haoyu, et al.
Published: (2024)

Assertion Detection Large Language Model In-context Learning LoRA Fine-tuning
by: Ji, Yuelyu, et al.
Published: (2024)

TLoRA: Task-aware Low Rank Adaptation of Large Language Models
by: Lin, Weicheng, et al.
Published: (2026)

OLoRA: Orthonormal Low-Rank Adaptation of Large Language Models
by: Büyükakyüz, Kerim
Published: (2024)

Chain of LoRA: Efficient Fine-tuning of Language Models via Residual Learning
by: Xia, Wenhan, et al.
Published: (2024)

LoRASuite: Efficient LoRA Adaptation Across Large Language Model Upgrades
by: Li, Yanan, et al.
Published: (2025)

MSPLoRA: A Multi-Scale Pyramid Low-Rank Adaptation for Efficient Model Fine-Tuning
by: Zhao, Jiancheng, et al.
Published: (2025)

DiffLoRA: Differential Low-Rank Adapters for Large Language Models
by: Misrahi, Alexandre, et al.
Published: (2025)

ID-LoRA: Efficient Low-Rank Adaptation Inspired by Matrix Interpolative Decomposition
by: Ma, Xindian, et al.
Published: (2026)

AFLoRA: Adaptive Freezing of Low Rank Adaptation in Parameter Efficient Fine-Tuning of Large Models
by: Liu, Zeyu, et al.
Published: (2024)

LoRA-Leak: Membership Inference Attacks Against LoRA Fine-tuned Language Models
by: Ran, Delong, et al.
Published: (2025)

ShapLoRA: Allocation of Low-rank Adaption on Large Language Models via Shapley Value Inspired Importance Estimation
by: Zhao, Yi, et al.
Published: (2026)

LoRA-GA: Low-Rank Adaptation with Gradient Approximation
by: Wang, Shaowen, et al.
Published: (2024)

LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models
by: Yang, Yifan, et al.
Published: (2024)

Why LoRA Fails to Forget: Regularized Low-Rank Adaptation Against Backdoors in Language Models
by: Luong, Hoang-Chau, et al.
Published: (2026)

TLoRA+: A Low-Rank Parameter-Efficient Fine-Tuning Method for Large Language Models
by: Cao, Yarui, et al.
Published: (2026)

LoRA-SP: Streamlined Partial Parameter Adaptation for Resource-Efficient Fine-Tuning of Large Language Models
by: Wu, Yichao, et al.
Published: (2024)

MatryoshkaLoRA: Learning Accurate Hierarchical Low-Rank Representations for LLM Fine-Tuning
by: Modoranu, Ionut-Vlad, et al.
Published: (2026)

Robust and Efficient Fine-tuning of LLMs with Bayesian Reparameterization of Low-Rank Adaptation
by: Sengupta, Ayan, et al.
Published: (2024)

AdaMoLE: Fine-Tuning Large Language Models with Adaptive Mixture of Low-Rank Adaptation Experts
by: Liu, Zefang, et al.
Published: (2024)