Cover Image

Balanced LoRA: Removing Parameter Invariance to Accelerate Convergence

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Castin, Valérie, Nadjahi, Kimia, Ablin, Pierre, Peyré, Gabriel
Format:	Preprint
Published:	2026
Subjects:	Machine Learning 68T07
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Unified Perspective on the Dynamics of Deep Transformers
by: Castin, Valérie, et al.
Published: (2025)

How Smooth Is Attention?
by: Castin, Valérie, et al.
Published: (2023)

FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
by: Zhang, Haoran, et al.
Published: (2026)

A Generalization Bound for a Family of Implicit Networks
by: Fung, Samy Wu, et al.
Published: (2024)

Evaluating Embedding Generalization: How LLMs, LoRA, and SLERP Shape Representational Geometry
by: Kabane, Siyaxolisa
Published: (2025)

A LoRA-Based Approach to Fine-Tuning LLMs for Educational Guidance in Resource-Constrained Settings
by: Hosen, Md Millat
Published: (2025)

On The Convergence of Euler Discretization of Finite-Time Convergent Gradient Flows
by: Zhang, Siqi, et al.
Published: (2020)

LoRA meets Riemannion: Muon Optimizer for Parametrization-independent Low-Rank Adapters
by: Bogachev, Vladimir, et al.
Published: (2025)

Exploring Efficient Learning of Small BERT Networks with LoRA and DoRA
by: Frees, Daniel, et al.
Published: (2025)

QuAILoRA: Quantization-Aware Initialization for LoRA
by: Lawton, Neal, et al.
Published: (2024)

Super-fast Rates of Convergence for Neural Network Classifiers under the Hard Margin Condition
by: Tepakbong, Nathanael, et al.
Published: (2025)

Influence-Inspired Spectral Rotations for Extreme Low-Bit LLM Quantization
by: Pavlov, Gorgi
Published: (2026)

From Features to Graphs: Exploring Graph Structures and Pairwise Interactions via GNNs
by: Yamchote, Phaphontee, et al.
Published: (2025)

The Inhibitor: ReLU and Addition-Based Attention for Efficient Transformers under Fully Homomorphic Encryption on the Torus
by: Brännvall, Rickard, et al.
Published: (2023)

Convergence of Continuous Normalizing Flows for Learning Probability Distributions
by: Gao, Yuan, et al.
Published: (2024)

The Boundaries of Verifiable Accuracy, Robustness, and Generalisation in Deep Learning
by: Bastounis, Alexander, et al.
Published: (2023)

Global Convergence of Sampling-Based Nonconvex Optimization through Diffusion-Style Smoothing
by: Yi, Zeji, et al.
Published: (2026)

Tadpole: Autoencoders as Foundation Models for 3D PDEs with Online Learning
by: Liu, Qiang, et al.
Published: (2026)

Flow matching on homogeneous spaces
by: Ruscelli, Francesco
Published: (2026)

Training-Free Generative Sampling via Moment-Matched Score Smoothing
by: Yao, Zhenyu, et al.
Published: (2026)

Generative Design of Ship Propellers using Conditional Flow Matching
by: Kruger, Patrick, et al.
Published: (2026)

Beyond the Class Subspace: Teacher-Guided Training for Reliable Out-of-Distribution Detection in Single-Domain Models
by: Yang, Hong, et al.
Published: (2026)

TabPFN for Zero-shot Parametric Engineering Design Generation
by: Wang, Ke, et al.
Published: (2026)

Adynamical systems view of training generativemodels and the memorization phenomenon
by: Athreya, Siva, et al.
Published: (2026)

Wave-Attractor-Tree: A Hierarchical Binary Tree Reduction Architecture for Efficient Sequence Modeling
by: Berezkin, Igor
Published: (2026)

Graph-Conditional Flow Matching for Relational Data Generation
by: Scassola, Davide, et al.
Published: (2025)

Feature Learning Beyond the Edge of Stability
by: Terjék, Dávid
Published: (2025)

ConFIG: Towards Conflict-free Training of Physics Informed Neural Networks
by: Liu, Qiang, et al.
Published: (2024)

Optimizing Basis Function Selection in Constructive Wavelet Neural Networks and Its Applications
by: Huang, Dunsheng, et al.
Published: (2025)

A Teacher-Student Perspective on the Dynamics of Learning Near the Optimal Point
by: Couto, Carlos, et al.
Published: (2025)

A Comprehensive View of Personalized Federated Learning on Heterogeneous Clinical Datasets
by: Tavakoli, Fatemeh, et al.
Published: (2023)

Closed-Form Feedback-Free Learning with Forward Projection
by: O'Shea, Robert, et al.
Published: (2025)

Markov Chain Estimation with In-Context Learning
by: Lepage, Simon, et al.
Published: (2025)

Structured Knowledge Accumulation: The Principle of Entropic Least Action in Forward-Only Neural Learning
by: Quantiota, Bouarfa Mahi
Published: (2025)

Deep Learning and Transfer Learning Architectures for English Premier League Player Performance Forecasting
by: Frees, Daniel, et al.
Published: (2024)

Improving Fairness and Mitigating MADness in Generative Models
by: Mayer, Paul, et al.
Published: (2024)

Cooperative Multi-Agent Deep Reinforcement Learning in Content Ranking Optimization
by: Qin, Zhou, et al.
Published: (2024)

Stability Analysis of Equivariant Convolutional Representations Through The Lens of Equivariant Multi-layered CKNs
by: Chowdhury, Soutrik Roy
Published: (2024)

Iterative Orthogonalization Scaling Laws
by: Selvaraj, Devan
Published: (2025)

MLPs at the EOC: Spectrum of the NTK
by: Terjék, Dávid, et al.
Published: (2025)