Cover Image

LoRA Training Provably Converges to a Low-Rank Global Minimum or It Fails Loudly (But it Probably Won't Fail)

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kim, Junsu, Kim, Jaeyeon, Ryu, Ernest K.
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LoRA Training in the NTK Regime has No Spurious Local Minima
by: Jang, Uijeong, et al.
Published: (2024)

AuroRA: Breaking Low-Rank Bottleneck of LoRA with Nonlinear Mapping
by: Dong, Haonan, et al.
Published: (2025)

Convergent Stochastic Training of Attention and Understanding LoRA
by: Sun, Zhengkai, et al.
Published: (2026)

LoRA-GA: Low-Rank Adaptation with Gradient Approximation
by: Wang, Shaowen, et al.
Published: (2024)

On the Convergence Rate of LoRA Gradient Descent
by: Mu, Siqiao, et al.
Published: (2025)

Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training
by: Zhang, Chengqian, et al.
Published: (2026)

LoRA-Pro: Are Low-Rank Adapters Properly Optimized?
by: Wang, Zhengbo, et al.
Published: (2024)

LoRA+: Efficient Low Rank Adaptation of Large Models
by: Hayou, Soufiane, et al.
Published: (2024)

C-LoRA: Continual Low-Rank Adaptation for Pre-trained Models
by: Zhang, Xin, et al.
Published: (2025)

Stable-LoRA: Stabilizing Feature Learning of Low-Rank Adaptation
by: Wu, Yize, et al.
Published: (2026)

Flat-LoRA: Low-Rank Adaptation over a Flat Loss Landscape
by: Li, Tao, et al.
Published: (2024)

CopRA: A Progressive LoRA Training Strategy
by: Zhuang, Zhan, et al.
Published: (2024)

Run LoRA Run: Faster and Lighter LoRA Implementations
by: Cherniuk, Daria, et al.
Published: (2023)

MTL-LoRA: Low-Rank Adaptation for Multi-Task Learning
by: Yang, Yaming, et al.
Published: (2024)

SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learning
by: Wu, Yichen, et al.
Published: (2025)

Bernoulli-LoRA: A Theoretical Framework for Randomized Low-Rank Adaptation
by: Sokolov, Igor, et al.
Published: (2025)

GeLoRA: Geometric Adaptive Ranks For Efficient LoRA Fine-tuning
by: Ed-dib, Abdessalam, et al.
Published: (2024)

Simple Drop-in LoRA Conditioning on Attention Layers Will Improve Your Diffusion Model
by: Choi, Joo Young, et al.
Published: (2024)

Rethinking the Rank Threshold for LoRA Fine-Tuning
by: Park, Juneyoung
Published: (2026)

HRP: High-Rank Preheating for Superior LoRA Initialization
by: Chen, Yuzhu, et al.
Published: (2025)

ALTO: Adaptive LoRA Tuning and Orchestration for Heterogeneous LoRA Training Workloads
by: Zuo, Jingwei, et al.
Published: (2026)

Improving Fisher Information Estimation and Efficiency for LoRA-based LLM Unlearning
by: Kim, Yejin, et al.
Published: (2025)

FiLoRA: Focus-and-Ignore LoRA for Controllable Feature Reliance
by: Chung, Hyunsuk, et al.
Published: (2026)

HyperAdaLoRA: Accelerating LoRA Rank Allocation During Training via Hypernetworks without Sacrificing Performance
by: Zhang, Hao, et al.
Published: (2025)

Clip-Low Increases Entropy and Clip-High Decreases Entropy in Reinforcement Learning of Large Language Models
by: Park, Jaesung R., et al.
Published: (2025)

C-LoRA: Contextual Low-Rank Adaptation for Uncertainty Estimation in Large Language Models
by: Rahmati, Amir Hossein, et al.
Published: (2025)

D2-LoRA: A Synergistic Approach to Differential and Directional Low-Rank Adaptation
by: Fujisawa, Nozomu, et al.
Published: (2026)

LoREnc: Low-Rank Encryption for Securing Foundation Models and LoRA Adapters
by: Ahn, Beomjin, et al.
Published: (2026)

LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters
by: Bałazy, Klaudia, et al.
Published: (2024)

Riemannian Optimization for LoRA on the Stiefel Manifold
by: Park, Juneyoung, et al.
Published: (2025)

HiP-LoRA: Budgeted Spectral Plasticity for Robust Low-Rank Adaptation
by: Chen, Lixian, et al.
Published: (2026)

CE-LoRA: Computation-Efficient LoRA Fine-Tuning for Language Models
by: Chen, Guanduo, et al.
Published: (2025)

LoRA Diffusion: Zero-Shot LoRA Synthesis for Diffusion Model Personalization
by: Smith, Ethan, et al.
Published: (2024)

tLoRA: Efficient Multi-LoRA Training with Elastic Shared Super-Models
by: Li, Kevin, et al.
Published: (2026)

FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
by: Zhang, Haoran, et al.
Published: (2026)

Balanced LoRA: Removing Parameter Invariance to Accelerate Convergence
by: Castin, Valérie, et al.
Published: (2026)

Block-Diagonal LoRA for Eliminating Communication Overhead in Tensor Parallel LoRA Serving
by: Wang, Xinyu, et al.
Published: (2025)

FIM-LoRA: Task-Informative Rank Allocation for LoRA via Calibration-Time Gradient-Variance Estimation
by: Sathyavageeswaran, Ramakrishnan
Published: (2026)

FM-LoRA: Factorized Low-Rank Meta-Prompting for Continual Learning
by: Yu, Xiaobing, et al.
Published: (2025)

S-LoRA: Serving Thousands of Concurrent LoRA Adapters
by: Sheng, Ying, et al.
Published: (2023)