Cover Image

MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Hanqing, Li, Yixia, Wang, Shuo, Chen, Guanhua, Chen, Yun
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuning
by: Zhang, Jingfan, et al.
Published: (2024)

LoRA-Flow: Dynamic LoRA Fusion for Large Language Models in Generative Tasks
by: Wang, Hanqing, et al.
Published: (2024)

Robust LLM Unlearning Against Relearning Attacks: The Minor Components in Representations Matter
by: Xiao, Zeguan, et al.
Published: (2026)

PACIT: Unlocking the Power of Examples for Better In-Context Instruction Tuning
by: Xue, Tianci, et al.
Published: (2023)

SeTAR: Out-of-Distribution Detection with Selective Low-Rank Approximation
by: Li, Yixia, et al.
Published: (2024)

ProtoBERT-LoRA: Parameter-Efficient Prototypical Finetuning for Immunotherapy Study Identification
by: Zhang, Shijia, et al.
Published: (2025)

Not All LoRA Parameters Are Essential: Insights on Inference Necessity
by: Chen, Guanhua, et al.
Published: (2025)

Representation-Guided Parameter-Efficient LLM Unlearning
by: Xiao, Zeguan, et al.
Published: (2026)

ImPart: Importance-Aware Delta-Sparsification for Improved Model Compression and Merging in LLMs
by: Yang, Yan, et al.
Published: (2025)

LoRA-drop: Efficient LoRA Parameter Pruning based on Output Evaluation
by: Zhou, Hongyun, et al.
Published: (2024)

Unveiling Over-Memorization in Finetuning LLMs for Reasoning Tasks
by: Ruan, Zhiwen, et al.
Published: (2025)

Efficient Multilingual ASR Finetuning via LoRA Language Experts
by: Li, Jiahong, et al.
Published: (2025)

LoRA-Drop: Temporal LoRA Decoding for Efficient LLM Inference
by: Rajabzadeh, Hossein, et al.
Published: (2026)

Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuning
by: Ruan, Zhiwen, et al.
Published: (2025)

LoRA-PAR: A Flexible Dual-System LoRA Partitioning Approach to Efficient LLM Fine-Tuning
by: Huang, Yining, et al.
Published: (2025)

Flexi-LoRA with Input-Adaptive Ranks: Efficient Finetuning for Speech and Reasoning Tasks
by: Li, Zongqian, et al.
Published: (2026)

PLoP: Precise LoRA Placement for Efficient Finetuning of Large Models
by: Hayou, Soufiane, et al.
Published: (2025)

LayAlign: Enhancing Multilingual Reasoning in Large Language Models via Layer-Wise Adaptive Fusion and Alignment Strategy
by: Ruan, Zhiwen, et al.
Published: (2025)

The Impact of Initialization on LoRA Finetuning Dynamics
by: Hayou, Soufiane, et al.
Published: (2024)

NeuroLoRA: Context-Aware Neuromodulation for Parameter-Efficient Multi-Task Adaptation
by: Yang, Yuxin, et al.
Published: (2026)

Sparse Memory Finetuning as a Low-Forgetting Alternative to LoRA and Full Finetuning
by: Gupta, Prakhar, et al.
Published: (2026)

Learning Rate Scaling across LoRA Ranks and Transfer to Full Finetuning
by: Chen, Nan, et al.
Published: (2026)

How LoRA Remembers? A Parametric Memory Law for LLM Finetuning
by: Xu, Ziwen, et al.
Published: (2026)

G2: Guided Generation for Enhanced Output Diversity in LLMs
by: Ruan, Zhiwen, et al.
Published: (2025)

Accurate LoRA-Finetuning Quantization of LLMs via Information Retention
by: Qin, Haotong, et al.
Published: (2024)

VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks
by: Li, Yang, et al.
Published: (2024)

LoRA-Whisper: Parameter-Efficient and Extensible Multilingual ASR
by: Song, Zheshu, et al.
Published: (2024)

MiSS: Revisiting the Trade-off in LoRA with an Efficient Shard-Sharing Structure
by: Kang, Jiale, et al.
Published: (2024)

OrthoGeoLoRA: Geometric Parameter-Efficient Fine-Tuning for Structured Social Science Concept Retrieval on theWeb
by: Wang, Zeqiang, et al.
Published: (2026)

CA-LoRA: Adapting Existing LoRA for Compressed LLMs to Enable Efficient Multi-Tasking on Personal Devices
by: Zhao, Weilin, et al.
Published: (2023)

PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization
by: Meng, Xiangdi, et al.
Published: (2024)

Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment
by: Fan, Chenghao, et al.
Published: (2025)

LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning
by: Guo, Han, et al.
Published: (2023)

LoraMap: Harnessing the Power of LoRA Connections
by: Park, Hyeryun, et al.
Published: (2024)

HydraLoRA: An Asymmetric LoRA Architecture for Efficient Fine-Tuning
by: Tian, Chunlin, et al.
Published: (2024)

Recover-to-Forget: Gradient Reconstruction from LoRA for Efficient LLM Unlearning
by: Liu, Yezi, et al.
Published: (2025)

OPLoRA: Orthogonal Projection LoRA Prevents Catastrophic Forgetting during Parameter-Efficient Fine-Tuning
by: Xiong, Yifeng, et al.
Published: (2025)

Towards Fair and Comprehensive Evaluation of Routers in Collaborative LLM Systems
by: Wu, Wanxing, et al.
Published: (2026)

SplitLoRA: A Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
by: Lin, Zheng, et al.
Published: (2024)

LUNE: Efficient LLM Unlearning via LoRA Fine-Tuning with Negative Examples
by: Liu, Yezi, et al.
Published: (2025)