Image de couverture de livre

Understanding the Learning Dynamics of LoRA: A Gradient Flow Perspective on Low-Rank Adaptation in Matrix Factorization

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Xu, Ziqing, Min, Hancheng, MacDonald, Lachlan Ewen, Luo, Jinqi, Tarmoun, Salma, Mallada, Enrique, Vidal, Rene
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Convergence Rates for Gradient Descent on the Edge of Stability in Overparametrised Least Squares
par: MacDonald, Lachlan Ewen, et autres
Publié: (2025)

A Local Polyak-Lojasiewicz and Descent Lemma of Gradient Descent For Overparametrized Linear Models
par: Xu, Ziqing, et autres
Publié: (2025)

Centre manifold theorem for maps along manifolds of fixed points
par: MacDonald, Lachlan Ewen
Publié: (2026)

Understanding Incremental Learning with Closed-form Solution to Gradient Flow on Overparamerterized Matrix Factorization
par: Min, Hancheng, et autres
Publié: (2025)

LoRA-GA: Low-Rank Adaptation with Gradient Approximation
par: Wang, Shaowen, et autres
Publié: (2024)

Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization
par: Min, Hancheng, et autres
Publié: (2023)

ID-LoRA: Efficient Low-Rank Adaptation Inspired by Matrix Interpolative Decomposition
par: Ma, Xindian, et autres
Publié: (2026)

Null-LoRA: Low-Rank Adaptation on Null Space
par: Zhang, Yi, et autres
Publié: (2025)

LoRA+: Efficient Low Rank Adaptation of Large Models
par: Hayou, Soufiane, et autres
Publié: (2024)

Chern-Weil theory for Haefliger-singular foliations
par: MacDonald, Lachlan Ewen, et autres
Publié: (2021)

Disintegration theorem for multifunctions, with applications to empirical Wasserstein distances and average-case statistical bounds
par: Fill, James Allen, et autres
Publié: (2025)

Stable-LoRA: Stabilizing Feature Learning of Low-Rank Adaptation
par: Wu, Yize, et autres
Publié: (2026)

1LoRA: Summation Compression for Very Low-Rank Adaptation
par: Quercia, Alessio, et autres
Publié: (2025)

MTL-LoRA: Low-Rank Adaptation for Multi-Task Learning
par: Yang, Yaming, et autres
Publié: (2024)

Rank-Accuracy Trade-off for LoRA: A Gradient-Flow Analysis
par: Rushka, Michael, et autres
Publié: (2026)

Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language Models
par: Lin, Moule, et autres
Publié: (2026)

C-LoRA: Continual Low-Rank Adaptation for Pre-trained Models
par: Zhang, Xin, et autres
Publié: (2025)

LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters
par: Bałazy, Klaudia, et autres
Publié: (2024)

Bernoulli-LoRA: A Theoretical Framework for Randomized Low-Rank Adaptation
par: Sokolov, Igor, et autres
Publié: (2025)

PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization
par: Meng, Xiangdi, et autres
Publié: (2024)

FIM-LoRA: Task-Informative Rank Allocation for LoRA via Calibration-Time Gradient-Variance Estimation
par: Sathyavageeswaran, Ramakrishnan
Publié: (2026)

Computational Limits of Low-Rank Adaptation (LoRA) Fine-Tuning for Transformer Models
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2024)

HiP-LoRA: Budgeted Spectral Plasticity for Robust Low-Rank Adaptation
par: Chen, Lixian, et autres
Publié: (2026)

PC-LoRA: Low-Rank Adaptation for Progressive Model Compression with Knowledge Distillation
par: Hwang, Injoon, et autres
Publié: (2024)

Flat-LoRA: Low-Rank Adaptation over a Flat Loss Landscape
par: Li, Tao, et autres
Publié: (2024)

SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learning
par: Wu, Yichen, et autres
Publié: (2025)

ABM-LoRA: Activation Boundary Matching for Fast Convergence in Low-Rank Adaptation
par: Lee, Dongha, et autres
Publié: (2025)

AuroRA: Breaking Low-Rank Bottleneck of LoRA with Nonlinear Mapping
par: Dong, Haonan, et autres
Publié: (2025)

G2LoRA: Gradient Orthogonal Low-Rank Adaptation Framework for Graph Continual Learning on Text-Attributed Graphs
par: Wang, Yuhan, et autres
Publié: (2026)

D2-LoRA: A Synergistic Approach to Differential and Directional Low-Rank Adaptation
par: Fujisawa, Nozomu, et autres
Publié: (2026)

LoRA-DA: Data-Aware Initialization for Low-Rank Adaptation via Asymptotic Analysis
par: Zhang, Qingyue, et autres
Publié: (2025)

Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training
par: Zhang, Chengqian, et autres
Publié: (2026)

Online-LoRA: Task-free Online Continual Learning via Low Rank Adaptation
par: Wei, Xiwen, et autres
Publié: (2024)

CL-LoRA: Continual Low-Rank Adaptation for Rehearsal-Free Class-Incremental Learning
par: He, Jiangpeng, et autres
Publié: (2025)

C-LoRA: Contextual Low-Rank Adaptation for Uncertainty Estimation in Large Language Models
par: Rahmati, Amir Hossein, et autres
Publié: (2025)

Laplacian-LoRA: Delaying Oversmoothing in Deep GCNs via Spectral Low-Rank Adaptation
par: Alisetti, Sai Vamsi
Publié: (2026)

UIR-LoRA: Achieving Universal Image Restoration through Multiple Low-Rank Adaptation
par: Zhang, Cheng, et autres
Publié: (2024)

Randomized Asymmetric Chain of LoRA: The First Meaningful Theoretical Framework for Low-Rank Adaptation
par: Malinovsky, Grigory, et autres
Publié: (2024)

LoRA-Pro: Are Low-Rank Adapters Properly Optimized?
par: Wang, Zhengbo, et autres
Publié: (2024)

SingLoRA: Low Rank Adaptation Using a Single Matrix
par: Bensaïd, David, et autres
Publié: (2025)