Cover Image

LoRA-MGPO: Mitigating Double Descent in Low-Rank Adaptation via Momentum-Guided Perturbation Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chang, Yupeng, Guo, Chenlu, Chang, Yi, Wu, Yuan
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

BA-LoRA: Bias-Alleviating Low-Rank Adaptation to Mitigate Catastrophic Inheritance in Large Language Models
by: Chang, Yupeng, et al.
Published: (2024)

NLoRA: Nyström-Initiated Low-Rank Adaptation for Large Language Models
by: Guo, Chenlu, et al.
Published: (2025)

PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization
by: Meng, Xiangdi, et al.
Published: (2024)

LoRA-GA: Low-Rank Adaptation with Gradient Approximation
by: Wang, Shaowen, et al.
Published: (2024)

LoRA+: Efficient Low Rank Adaptation of Large Models
by: Hayou, Soufiane, et al.
Published: (2024)

Transfer-Prompting: Enhancing Cross-Task Adaptation in Large Language Models via Dual-Stage Prompts Optimization
by: Chang, Yupeng, et al.
Published: (2025)

MTL-LoRA: Low-Rank Adaptation for Multi-Task Learning
by: Yang, Yaming, et al.
Published: (2024)

ID-LoRA: Efficient Low-Rank Adaptation Inspired by Matrix Interpolative Decomposition
by: Ma, Xindian, et al.
Published: (2026)

LoRA-Pro: Are Low-Rank Adapters Properly Optimized?
by: Wang, Zhengbo, et al.
Published: (2024)

Align, Don't Divide: Revisiting the LoRA Architecture in Multi-Task Learning
by: Liu, Jinda, et al.
Published: (2025)

LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters
by: Bałazy, Klaudia, et al.
Published: (2024)

HiP-LoRA: Budgeted Spectral Plasticity for Robust Low-Rank Adaptation
by: Chen, Lixian, et al.
Published: (2026)

Why LoRA Fails to Forget: Regularized Low-Rank Adaptation Against Backdoors in Language Models
by: Luong, Hoang-Chau, et al.
Published: (2026)

MeTA-LoRA: Data-Efficient Multi-Task Fine-Tuning for Large Language Models
by: Cheng, Bo, et al.
Published: (2025)

DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models
by: Deng, Guanzhi, et al.
Published: (2026)

$D^2LoRA$: Data-Driven LoRA Initialization for Low Resource Tasks
by: SeraJ, Javad, et al.
Published: (2025)

AFA-LoRA: Enabling Non-Linear Adaptations in LoRA with Activation Function Annealing
by: Li, Jiacheng, et al.
Published: (2025)

BeamLoRA: Beam-Constraint Low-Rank Adaptation
by: Gu, Naibin, et al.
Published: (2025)

GuiLoMo: Allocating Expert Number and Rank for LoRA-MoE via Bilevel Optimization with GuidedSelection Vectors
by: Zhang, Hengyuan, et al.
Published: (2025)

LoRA-FA: Efficient and Effective Low Rank Representation Fine-tuning
by: Zhang, Longteng, et al.
Published: (2023)

Recover-LoRA: Data-Free Accuracy Recovery of Degraded Language Models via Low-Rank Adaptation
by: Das, Devleena, et al.
Published: (2025)

A Lightweight Framework for Trigger-Guided LoRA-Based Self-Adaptation in LLMs
by: Wei, Jiacheng, et al.
Published: (2025)

ChunkWise LoRA: Adaptive Sequence Partitioning for Memory-Efficient Low-Rank Adaptation and Accelerated LLM Inference
by: Thakkar, Ketan, et al.
Published: (2026)

Dual LoRA: Enhancing LoRA with Magnitude and Direction Updates
by: Xu, Yixing, et al.
Published: (2025)

RST-LoRA: A Discourse-Aware Low-Rank Adaptation for Long Document Abstractive Summarization
by: Liu, Dongqi, et al.
Published: (2024)

FIM-LoRA: Task-Informative Rank Allocation for LoRA via Calibration-Time Gradient-Variance Estimation
by: Sathyavageeswaran, Ramakrishnan
Published: (2026)

Null-LoRA: Low-Rank Adaptation on Null Space
by: Zhang, Yi, et al.
Published: (2025)

LoRA-Drop: Temporal LoRA Decoding for Efficient LLM Inference
by: Rajabzadeh, Hossein, et al.
Published: (2026)

LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA Optimization
by: Yen, Jui-Nan, et al.
Published: (2024)

L1RA: Dynamic Rank Assignment in LoRA Fine-Tuning
by: Singh, Raul, et al.
Published: (2025)

Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation
by: Tang, Haozhan, et al.
Published: (2026)

ALTER: Asymmetric LoRA for Token-Entropy-Guided Unlearning of LLMs
by: Chen, Xunlei, et al.
Published: (2026)

Red-Bandit: Test-Time Adaptation for LLM Red-Teaming via Bandit-Guided LoRA Experts
by: Ziakas, Christos, et al.
Published: (2025)

LoRA-Gen: Specializing Large Language Model via Online LoRA Generation
by: Xiao, Yicheng, et al.
Published: (2025)

HyperAdaLoRA: Accelerating LoRA Rank Allocation During Training via Hypernetworks without Sacrificing Performance
by: Zhang, Hao, et al.
Published: (2025)

Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment
by: Fan, Chenghao, et al.
Published: (2025)

Mixture of LoRA Experts
by: Wu, Xun, et al.
Published: (2024)

Vision as LoRA
by: Wang, Han, et al.
Published: (2025)

Octavius: Mitigating Task Interference in MLLMs via LoRA-MoE
by: Chen, Zeren, et al.
Published: (2023)

DenseLoRA: Dense Low-Rank Adaptation of Large Language Models
by: Mu, Lin, et al.
Published: (2025)