Imagen de Portada

Subspace Optimization for Large Language Models with Convergence Guarantees

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	He, Yutong, Li, Pengrui, Hu, Yipeng, Chen, Chuyan, Yuan, Kun
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Optimization and Control
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Greedy Low-Rank Gradient Compression for Distributed Learning with Convergence Guarantees
por: Chen, Chuyan, et al.
Publicado: (2025)

From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees
por: Xie, Shengping, et al.
Publicado: (2025)

Adam-family Methods for Nonsmooth Optimization with Convergence Guarantees
por: Xiao, Nachuan, et al.
Publicado: (2023)

Lean and Mean Adaptive Optimization via Subset-Norm and Subspace-Momentum with Convergence Guarantees
por: Nguyen, Thien Hang, et al.
Publicado: (2024)

Stochastic Compositional Minimax Optimization with Provable Convergence Guarantees
por: Deng, Yuyang, et al.
Publicado: (2024)

Convergence and Complexity Guarantee for Inexact First-order Riemannian Optimization Algorithms
por: Li, Yuchen, et al.
Publicado: (2024)

Unbiased Compression Saves Communication in Distributed Optimization: When and How Much?
por: He, Yutong, et al.
Publicado: (2023)

Global Convergence of Iteratively Reweighted Least Squares for Robust Subspace Recovery
por: Lerman, Gilad, et al.
Publicado: (2025)

Memory-Reduced Meta-Learning with Guaranteed Convergence
por: Yang, Honglin, et al.
Publicado: (2024)

MAP Estimation with Denoisers: Convergence Rates and Guarantees
por: Pesme, Scott, et al.
Publicado: (2025)

Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance
por: Zhang, Qi, et al.
Publicado: (2024)

Convergence of Spectral Descent for Non-smooth Optimization
por: Yang, Yixuan, et al.
Publicado: (2026)

Learning Over-Relaxation Policies for ADMM with Convergence Guarantees
por: Lin, Junan, et al.
Publicado: (2026)

On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization
por: Zhou, Dongruo, et al.
Publicado: (2018)

Achieving Near-Optimal Convergence for Distributed Minimax Optimization with Adaptive Stepsizes
por: Huang, Yan, et al.
Publicado: (2024)

On the Convergence of Stochastic Gradient Descent with Perturbed Forward-Backward Passes
por: Kong, Boao, et al.
Publicado: (2026)

Shuffling Heuristic in Variational Inequalities: Establishing New Convergence Guarantees
por: Medyakov, Daniil, et al.
Publicado: (2025)

Simple Stepsize for Quasi-Newton Methods with Global Convergence Guarantees
por: Agafonov, Artem, et al.
Publicado: (2025)

Optimal Guarantees for Algorithmic Reproducibility and Gradient Complexity in Convex Optimization
por: Zhang, Liang, et al.
Publicado: (2023)

Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression
por: He, Yutong, et al.
Publicado: (2023)

Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance
por: Oikonomou, Dimitris, et al.
Publicado: (2024)

A New First-Order Meta-Learning Algorithm with Convergence Guarantees
por: Chayti, El Mahdi, et al.
Publicado: (2024)

QLABGrad: a Hyperparameter-Free and Convergence-Guaranteed Scheme for Deep Learning
por: Fu, Minghan, et al.
Publicado: (2023)

Subspace Optimization for Efficient Federated Learning under Heterogeneous Data
por: Zhu, Shuchen, et al.
Publicado: (2026)

Group Projected Subspace Pursuit for Block Sparse Signal Reconstruction: Convergence Analysis and Applications
por: He, Roy Y., et al.
Publicado: (2024)

GNMR: Runtime Stability Control for Low-Precision Large Language Model Training
por: Kong, Boao, et al.
Publicado: (2026)

Optimization over Sparse Support-Preserving Sets: Two-Step Projection with Global Optimality Guarantees
por: de Vazelhes, William, et al.
Publicado: (2025)

Global Convergence Guarantees for Federated Policy Gradient Methods with Adversaries
por: Ganesh, Swetha, et al.
Publicado: (2024)

Krylov Cubic Regularized Newton: A Subspace Second-Order Method with Dimension-Free Convergence Rate
por: Jiang, Ruichen, et al.
Publicado: (2024)

Sharper Guarantees for Misspecified Kernelized Bandit Optimization
por: Maran, Davide, et al.
Publicado: (2026)

Towards Robust Learning to Optimize with Theoretical Guarantees
por: Song, Qingyu, et al.
Publicado: (2025)

A Convex-optimization-based Layer-wise Post-training Pruner for Large Language Models
por: Zhao, Pengxiang, et al.
Publicado: (2024)

High-Probability Convergence Guarantees of Decentralized SGD
por: Armacki, Aleksandar, et al.
Publicado: (2025)

Bilevel Models for Adversarial Learning and A Case Study
por: Zheng, Yutong, et al.
Publicado: (2025)

Optimization Hyper-parameter Laws for Large Language Models
por: Xie, Xingyu, et al.
Publicado: (2024)

Data-Driven Performance Guarantees for Classical and Learned Optimizers
por: Sambharya, Rajiv, et al.
Publicado: (2024)

On the Hardness of Meaningful Local Guarantees in Nonsmooth Nonconvex Optimization
por: Kornowski, Guy, et al.
Publicado: (2024)

Over-parameterised Shallow Neural Networks with Asymmetrical Node Scaling: Global Convergence Guarantees and Feature Learning
por: Caron, Francois, et al.
Publicado: (2023)

Local Linear Convergence of Infeasible Optimization with Orthogonal Constraints
por: Sun, Youbang, et al.
Publicado: (2024)

A Regularized Newton Method for Nonconvex Optimization with Global and Local Complexity Guarantees
por: Zhou, Yuhao, et al.
Publicado: (2025)