Cover Image

LoFT: Low-Rank Adaptation That Behaves Like Full Fine-Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tastan, Nurbek, Laskaridis, Stefanos, Takac, Martin, Nandakumar, Karthik, Horvath, Samuel
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MoSE: Mixture of Slimmable Experts for Efficient and Adaptive Language Models
by: Tastan, Nurbek, et al.
Published: (2026)

FedPeWS: Personalized Warmup via Subnetworks for Enhanced Heterogeneous Federated Learning
by: Tastan, Nurbek, et al.
Published: (2024)

Stochastic Self-Organization in Multi-Agent Systems
by: Tastan, Nurbek, et al.
Published: (2025)

Aequa: Fair Model Rewards in Collaborative Learning via Slimmable Networks
by: Tastan, Nurbek, et al.
Published: (2025)

CYCle: Choosing Your Collaborators Wisely to Enhance Collaborative Fairness in Decentralized Learning
by: Tastan, Nurbek, et al.
Published: (2025)

Federated Learning Can Find Friends That Are Advantageous
by: Tupitsa, Nazarii, et al.
Published: (2024)

A Framework for Double-Blind Federated Adaptation of Foundation Models
by: Tastan, Nurbek, et al.
Published: (2025)

Simple Stepsize for Quasi-Newton Methods with Global Convergence Guarantees
by: Agafonov, Artem, et al.
Published: (2025)

Bernoulli-LoRA: A Theoretical Framework for Randomized Low-Rank Adaptation
by: Sokolov, Igor, et al.
Published: (2025)

Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods
by: Veprikov, Andrey, et al.
Published: (2025)

Methods for Convex $(L_0,L_1)$-Smooth Optimization: Clipping, Acceleration, and Adaptivity
by: Gorbunov, Eduard, et al.
Published: (2024)

Methods with Local Steps and Random Reshuffling for Generally Smooth Non-Convex Federated Optimization
by: Demidovich, Yury, et al.
Published: (2024)

Randomized Asymmetric Chain of LoRA: The First Meaningful Theoretical Framework for Low-Rank Adaptation
by: Malinovsky, Grigory, et al.
Published: (2024)

FlexRank: Nested Low-Rank Knowledge Decomposition for Adaptive Model Deployment
by: Zaccone, Riccardo, et al.
Published: (2026)

Revisiting LocalSGD and SCAFFOLD: Improved Rates and Missing Analysis
by: Luo, Ruichen, et al.
Published: (2025)

Remove that Square Root: A New Efficient Scale-Invariant Version of AdaGrad
by: Choudhury, Sayantan, et al.
Published: (2024)

Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed
by: Chezhegov, Savelii, et al.
Published: (2024)

Redefining Contributions: Shapley-Driven Federated Learning
by: Tastan, Nurbek, et al.
Published: (2024)

Maestro: Uncovering Low-Rank Structures via Trainable Decomposition
by: Horvath, Samuel, et al.
Published: (2023)

Random-reshuffled SARAH does not need a full gradient computations
by: Beznosikov, Aleksandr, et al.
Published: (2021)

MOLM: Mixture of LoRA Markers
by: Fares, Samar, et al.
Published: (2025)

SPDMark: Selective Parameter Displacement for Robust Video Watermarking
by: Fares, Samar, et al.
Published: (2025)

Riemannian Preconditioned LoRA for Fine-Tuning Foundation Models
by: Zhang, Fangzhao, et al.
Published: (2024)

Gradient Clipping Beyond Vector Norms: A Spectral Approach for Matrix-Valued Parameters
by: Yukhimchuk, Alexander, et al.
Published: (2026)

Low-Rank Extragradient Method for Nonsmooth and Low-Rank Matrix Optimization Problems
by: Garber, Dan, et al.
Published: (2022)

Low-Rank Mirror-Prox for Nonsmooth and Low-Rank Matrix Optimization Problems
by: Garber, Dan, et al.
Published: (2022)

Where Does Warm-Up Come From? Adaptive Scheduling for Norm-Constrained Optimizers
by: Riabinin, Artem, et al.
Published: (2026)

SANIA: Polyak-type Optimization Framework Leads to Scale Invariant Stochastic Algorithms
by: Abdukhakimov, Farshed, et al.
Published: (2023)

Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
by: Choudhury, Sayantan, et al.
Published: (2026)

Gradient-Informed Monte Carlo Fine-Tuning of Diffusion Models for Low-Thrust Trajectory Design
by: Graebner, Jannik, et al.
Published: (2025)

Convergence of Clipped-SGD for Convex $(L_0,L_1)$-Smooth Optimization with Heavy-Tailed Noise
by: Chezhegov, Savelii, et al.
Published: (2025)

AdaBatchGrad: Combining Adaptive Batch Size and Adaptive Step Size
by: Ostroukhov, Petr, et al.
Published: (2024)

$ψ$DAG: Projected Stochastic Approximation Iteration for DAG Structure Learning
by: Ziu, Klea, et al.
Published: (2024)

An Overview of Low-Rank Structures in the Training and Adaptation of Large Models
by: Balzano, Laura, et al.
Published: (2025)

Compact Lifted Relaxations for Low-Rank Optimization
by: Cory-Wright, Ryan, et al.
Published: (2026)

Heaviside Low-Rank Support Matrix Machine
by: Xiu, Xianchao, et al.
Published: (2026)

Control Theoretic Approach to Fine-Tuning and Transfer Learning
by: Bayram, Erkan, et al.
Published: (2024)

Double Momentum and Error Feedback for Clipping with Fast Rates and Differential Privacy
by: Islamov, Rustem, et al.
Published: (2025)

Stochastic Gradient Methods with Preconditioned Updates
by: Sadiev, Abdurakhmon, et al.
Published: (2022)

Efficient Model-Free Exploration in Low-Rank MDPs
by: Mhammedi, Zakaria, et al.
Published: (2023)