Cover Image

Parameter-Efficient Subspace Optimization for LLM Fine-Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lou, Yuchen, Ye, Zeqi, Chen, Minshuo
Format:	Preprint
Published:	2025
Subjects:	Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Diffusion Model for Data-Driven Black-Box Optimization
by: Li, Zihao, et al.
Published: (2024)

Subspace Newton's Method for $\ell_0$-Regularized Optimization Problems with Box Constraint
by: Ye, Yuge, et al.
Published: (2025)

A Decomposition Framework for Nonlinear Nonconvex Two-Stage Optimization
by: Lou, Yuchen, et al.
Published: (2025)

Zero-Order Optimization for LLM Fine-Tuning via Learnable Direction Sampling
by: Parfenov, Valery, et al.
Published: (2026)

Efficient Tail-Aware Generative Optimization via Flow Model Fine-Tuning
by: Wang, Zifan, et al.
Published: (2026)

Augmenting Subspace Optimization Methods with Linear Bandits
by: Menickelly, Matt
Published: (2024)

Subspace Optimization for Efficient Federated Learning under Heterogeneous Data
by: Zhu, Shuchen, et al.
Published: (2026)

Min-Max Grassmannian Optimization for Online Subspace Tracking
by: Bharadwaj, Shreyas, et al.
Published: (2026)

Accelerate Solving Expensive Scheduling by Leveraging Economical Auxiliary Tasks
by: Li, Minshuo, et al.
Published: (2024)

On Some Versions of Subspace Optimization Methods with Inexact Gradient Information
by: Kuruzov, Ilya, et al.
Published: (2024)

Space Mapping Optimization using Neural Networks for Efficient Parameter Estimation
by: Kotecha, Dhruvil Kamleshkumar
Published: (2025)

Scalable Derivative-Free Optimization Algorithms with Low-Dimensional Subspace Techniques
by: Zhang, Zaikun
Published: (2025)

Random-Subspace Sequential Quadratic Programming for Constrained Zeroth-Order Optimization
by: Zhang, Runyu, et al.
Published: (2026)

Subspace Optimization for Large Language Models with Convergence Guarantees
by: He, Yutong, et al.
Published: (2024)

Randomized Subspace Derivative-Free Optimization with Quadratic Models and Second-Order Convergence
by: Cartis, Coralia, et al.
Published: (2024)

Accelerated ADMM: Automated Parameter Tuning and Improved Linear Convergence
by: Tavakoli, Meisam, et al.
Published: (2025)

Efficient Algorithms for A Class of Stochastic Hidden Convex Optimization and Its Applications in Network Revenue Management
by: Chen, Xin, et al.
Published: (2022)

Curvature-Adaptive Perturbation and Subspace Descent for Robust Saddle Point Escape in High-Dimensional Optimization
by: Katende, Ronald, et al.
Published: (2024)

Online Sparse Subspace Clustering
by: Madden, Liam, et al.
Published: (2019)

A Parameter-free Decentralized Algorithm for Composite Convex Optimization
by: Chen, Xiaokai, et al.
Published: (2025)

Damping Tuning Considering Random Disturbances Adopting Distributionally Robust Optimization
by: Wang, Yuhong, et al.
Published: (2025)

SUMO: Subspace-Aware Moment-Orthogonalization for Accelerating Memory-Efficient LLM Training
by: Refael, Yehonathan, et al.
Published: (2025)

Understanding Forgetting in LLM Supervised Fine-Tuning and Preference Learning -- A Convex Optimization Perspective
by: Fernando, Heshan, et al.
Published: (2024)

Learning Stabilizing Policies via an Unstable Subspace Representation
by: Toso, Leonardo F., et al.
Published: (2025)

A Parameter-Free Zeroth-Order Algorithm for Decentralized Stochastic Convex Optimization
by: Chen, Jiawei, et al.
Published: (2026)

Online Optimization with Unknown Time-varying Parameters
by: Tripathi, Shivanshu, et al.
Published: (2025)

A Noise-Aware Scalable Subspace Classical Optimizer for the Quantum Approximate Optimization Algorithm
by: Dzahini, Kwassi Joseph, et al.
Published: (2025)

An Efficient Stochastic Algorithm for Decentralized Nonconvex-Strongly-Concave Minimax Optimization
by: Chen, Lesi, et al.
Published: (2022)

Online Distributed Optimization with Clipped Stochastic Gradients: High Probability Bound of Regrets
by: Yang, Yuchen, et al.
Published: (2024)

Subspace Quasi-Newton Method with Gradient Approximation
by: Miyaishi, Taisei, et al.
Published: (2024)

A Hessian-Free Actor-Critic Algorithm for Bi-Level Reinforcement Learning with Applications to LLM Fine-Tuning
by: Zeng, Sihan, et al.
Published: (2026)

Efficient Algorithms for Sum-of-Minimum Optimization
by: Ding, Lisang, et al.
Published: (2024)

Maintenance Optimization for Asset Networks with Unknown Degradation Parameters
by: Verleijsdonk, Peter, et al.
Published: (2024)

Universal and Parameter-free Gradient Sliding for Composite Optimization
by: Wu, Yan, et al.
Published: (2026)

Sample-Efficient Counterfactual Tuning for Compressor Pressure Control
by: Guerrero, Margarita A., et al.
Published: (2025)

Online Subspace Learning on Flag Manifolds for System Identification
by: Jin, Dian, et al.
Published: (2025)

Parameter Optimization in Trajectory Planning via Differentiable Convex Programming
by: Xu, Ziqi, et al.
Published: (2025)

Combined Parameter and Shape Optimization of Electric Machines with Isogeometric Analysis
by: Wiesheu, Michael, et al.
Published: (2023)

Towards Parameter-free Distributed Optimization: a Port-Hamiltonian Approach
by: Aldana-López, Rodrigo, et al.
Published: (2024)

Random-Subspace Frank--Wolfe over Strongly Convex Sets
by: Poirion, Pierre-Louis, et al.
Published: (2026)