Copertina

Understanding the Learning Dynamics of LoRA: A Gradient Flow Perspective on Low-Rank Adaptation in Matrix Factorization

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Xu, Ziqing, Min, Hancheng, MacDonald, Lachlan Ewen, Luo, Jinqi, Tarmoun, Salma, Mallada, Enrique, Vidal, Rene
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Convergence Rates for Gradient Descent on the Edge of Stability in Overparametrised Least Squares
di: MacDonald, Lachlan Ewen, et al.
Pubblicazione: (2025)

A Local Polyak-Lojasiewicz and Descent Lemma of Gradient Descent For Overparametrized Linear Models
di: Xu, Ziqing, et al.
Pubblicazione: (2025)

Centre manifold theorem for maps along manifolds of fixed points
di: MacDonald, Lachlan Ewen
Pubblicazione: (2026)

Understanding Incremental Learning with Closed-form Solution to Gradient Flow on Overparamerterized Matrix Factorization
di: Min, Hancheng, et al.
Pubblicazione: (2025)

LoRA-GA: Low-Rank Adaptation with Gradient Approximation
di: Wang, Shaowen, et al.
Pubblicazione: (2024)

Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization
di: Min, Hancheng, et al.
Pubblicazione: (2023)

ID-LoRA: Efficient Low-Rank Adaptation Inspired by Matrix Interpolative Decomposition
di: Ma, Xindian, et al.
Pubblicazione: (2026)

Null-LoRA: Low-Rank Adaptation on Null Space
di: Zhang, Yi, et al.
Pubblicazione: (2025)

LoRA+: Efficient Low Rank Adaptation of Large Models
di: Hayou, Soufiane, et al.
Pubblicazione: (2024)

Chern-Weil theory for Haefliger-singular foliations
di: MacDonald, Lachlan Ewen, et al.
Pubblicazione: (2021)

Disintegration theorem for multifunctions, with applications to empirical Wasserstein distances and average-case statistical bounds
di: Fill, James Allen, et al.
Pubblicazione: (2025)

Stable-LoRA: Stabilizing Feature Learning of Low-Rank Adaptation
di: Wu, Yize, et al.
Pubblicazione: (2026)

1LoRA: Summation Compression for Very Low-Rank Adaptation
di: Quercia, Alessio, et al.
Pubblicazione: (2025)

MTL-LoRA: Low-Rank Adaptation for Multi-Task Learning
di: Yang, Yaming, et al.
Pubblicazione: (2024)

Rank-Accuracy Trade-off for LoRA: A Gradient-Flow Analysis
di: Rushka, Michael, et al.
Pubblicazione: (2026)

Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language Models
di: Lin, Moule, et al.
Pubblicazione: (2026)

C-LoRA: Continual Low-Rank Adaptation for Pre-trained Models
di: Zhang, Xin, et al.
Pubblicazione: (2025)

LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters
di: Bałazy, Klaudia, et al.
Pubblicazione: (2024)

Bernoulli-LoRA: A Theoretical Framework for Randomized Low-Rank Adaptation
di: Sokolov, Igor, et al.
Pubblicazione: (2025)

PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization
di: Meng, Xiangdi, et al.
Pubblicazione: (2024)

FIM-LoRA: Task-Informative Rank Allocation for LoRA via Calibration-Time Gradient-Variance Estimation
di: Sathyavageeswaran, Ramakrishnan
Pubblicazione: (2026)

Computational Limits of Low-Rank Adaptation (LoRA) Fine-Tuning for Transformer Models
di: Hu, Jerry Yao-Chieh, et al.
Pubblicazione: (2024)

HiP-LoRA: Budgeted Spectral Plasticity for Robust Low-Rank Adaptation
di: Chen, Lixian, et al.
Pubblicazione: (2026)

PC-LoRA: Low-Rank Adaptation for Progressive Model Compression with Knowledge Distillation
di: Hwang, Injoon, et al.
Pubblicazione: (2024)

Flat-LoRA: Low-Rank Adaptation over a Flat Loss Landscape
di: Li, Tao, et al.
Pubblicazione: (2024)

SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learning
di: Wu, Yichen, et al.
Pubblicazione: (2025)

ABM-LoRA: Activation Boundary Matching for Fast Convergence in Low-Rank Adaptation
di: Lee, Dongha, et al.
Pubblicazione: (2025)

AuroRA: Breaking Low-Rank Bottleneck of LoRA with Nonlinear Mapping
di: Dong, Haonan, et al.
Pubblicazione: (2025)

G2LoRA: Gradient Orthogonal Low-Rank Adaptation Framework for Graph Continual Learning on Text-Attributed Graphs
di: Wang, Yuhan, et al.
Pubblicazione: (2026)

D2-LoRA: A Synergistic Approach to Differential and Directional Low-Rank Adaptation
di: Fujisawa, Nozomu, et al.
Pubblicazione: (2026)

LoRA-DA: Data-Aware Initialization for Low-Rank Adaptation via Asymptotic Analysis
di: Zhang, Qingyue, et al.
Pubblicazione: (2025)

Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training
di: Zhang, Chengqian, et al.
Pubblicazione: (2026)

Online-LoRA: Task-free Online Continual Learning via Low Rank Adaptation
di: Wei, Xiwen, et al.
Pubblicazione: (2024)

CL-LoRA: Continual Low-Rank Adaptation for Rehearsal-Free Class-Incremental Learning
di: He, Jiangpeng, et al.
Pubblicazione: (2025)

C-LoRA: Contextual Low-Rank Adaptation for Uncertainty Estimation in Large Language Models
di: Rahmati, Amir Hossein, et al.
Pubblicazione: (2025)

Laplacian-LoRA: Delaying Oversmoothing in Deep GCNs via Spectral Low-Rank Adaptation
di: Alisetti, Sai Vamsi
Pubblicazione: (2026)

UIR-LoRA: Achieving Universal Image Restoration through Multiple Low-Rank Adaptation
di: Zhang, Cheng, et al.
Pubblicazione: (2024)

Randomized Asymmetric Chain of LoRA: The First Meaningful Theoretical Framework for Low-Rank Adaptation
di: Malinovsky, Grigory, et al.
Pubblicazione: (2024)

LoRA-Pro: Are Low-Rank Adapters Properly Optimized?
di: Wang, Zhengbo, et al.
Pubblicazione: (2024)

SingLoRA: Low Rank Adaptation Using a Single Matrix
di: Bensaïd, David, et al.
Pubblicazione: (2025)