Cover Image

Low-rank Orthogonalization for Large-scale Matrix Optimization with Applications to Foundation Model Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	He, Chuan, Deng, Zhanwang, Lu, Zhaosong
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Optimization and Control 49M37, 90C30, 90C90
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Complexity of normalized stochastic first-order methods with momentum under heavy-tailed noise
by: He, Chuan, et al.
Published: (2025)

Accelerated stochastic first-order method for convex optimization under heavy-tailed noise
by: He, Chuan, et al.
Published: (2025)

Level Constrained First Order Methods for Function Constrained Optimization
by: Boob, Digvijay, et al.
Published: (2022)

Variational Poisson Denoising via Augmented Lagrangian Methods
by: Kanzow, Christian, et al.
Published: (2023)

Bias-Variance Trade-off for Clipped Stochastic First-Order Methods: From Bounded Variance to Infinite Mean
by: He, Chuan
Published: (2025)

Exploiting cone approximations in an augmented Lagrangian method for conic optimization
by: Fukuda, Mituhiro, et al.
Published: (2024)

Modified limited memory BFGS with displacement aggregation and its application to the largest eigenvalue problem
by: Sahu, Manish Kumar, et al.
Published: (2023)

Switching Point Optimization for Abstract Parabolic Equations
by: Buchheim, Christoph, et al.
Published: (2026)

A Newton-CG based barrier-augmented Lagrangian method for general nonconvex conic optimization
by: He, Chuan, et al.
Published: (2023)

A stochastic first-order method with multi-extrapolated momentum for highly smooth unconstrained optimization
by: He, Chuan
Published: (2024)

Retrospective Approximation Sequential Quadratic Programming for Stochastic Optimization with General Deterministic Nonlinear Constraints
by: Berahas, Albert S., et al.
Published: (2025)

Model-Driven Subspaces for Large-Scale Optimization with Local Approximation Strategy
by: He, Yitong, et al.
Published: (2025)

On the Convergence of Interior-Point Methods for Bound-Constrained Nonlinear Optimization Problems with Noise
by: Dezfulian, Shima, et al.
Published: (2024)

Trust-Region Method for Optimization of Set-Valued Maps Given by Finitely Many Functions
by: Ghosh, Suprova, et al.
Published: (2025)

A Proximal Stochastic Gradient Method with Adaptive Step Size and Variance Reduction for Convex Composite Optimization
by: Fang, Changjie, et al.
Published: (2025)

Asymptotic Convergence and Stability of Adaptive Gradient Methods in Smooth Non-convex Optimization
by: Jin, Ruinan, et al.
Published: (2026)

A cut-and-project perspective for linearized Bregman iterations
by: Dai, Yu-Hong, et al.
Published: (2024)

On the Convergence and Complexity of Proximal Gradient and Accelerated Proximal Gradient Methods under Adaptive Gradient Estimation
by: Bollapragada, Raghu, et al.
Published: (2025)

Alternating Gradient-Type Algorithm for Bilevel Optimization with Inexact Lower-Level Solutions via Moreau Envelope-based Reformulation
by: Bai, Xiaoning, et al.
Published: (2024)

On the resolution of $\ell_1$-norm minimization via a two-metric adaptive projection method
by: Wu, Hanju, et al.
Published: (2025)

Augmented Lagrangian methods for fully convex composite optimization
by: De Marchi, Alberto, et al.
Published: (2025)

Local Convergence Results for Sequential Quadratic Programming with Complementarity Constraints
by: Nurkanović, Armin
Published: (2026)

A Globally Convergent Method for Computing B-stationary Points of Mathematical Programs with Equilibrium Constraints
by: Nurkanović, Armin, et al.
Published: (2025)

First- and Second-Order Stochastic Adaptive Regularization with Cubics: High Probability Iteration and Sample Complexity
by: Scheinberg, Katya, et al.
Published: (2023)

Cubic Regularization Technique of the Newton Method for Vector Optimization
by: Ghosh, Debdas
Published: (2025)

The Grimmer--Shu--Wang Certificate and the Drori--Teboulle Minimax Constant-Stepsize Bound for $N\ge 3$
by: Zhang, Lixing
Published: (2026)

Flexible block-iterative analysis for the Frank-Wolfe algorithm
by: Braun, Gábor, et al.
Published: (2024)

An Interior-Point Algorithm for Continuous Nonlinearly Constrained Optimization with Noisy Function and Derivative Evaluations
by: Curtis, Frank E., et al.
Published: (2025)

Stochastic interior-point methods for smooth conic optimization with applications
by: He, Chuan, et al.
Published: (2024)

MGProx: A nonsmooth multigrid proximal gradient method with adaptive restriction for strongly convex optimization
by: Ang, Andersen, et al.
Published: (2023)

On Discrete Subproblems in Integer Optimal Control with Total Variation Regularization in Two Dimensions
by: Manns, Paul, et al.
Published: (2024)

Constraint Identification and Algorithm Stabilization for Degenerate Nonlinear Programs
by: Wright, Stephen J.
Published: (2000)

A Two Stepsize SQP Method for Nonlinear Equality Constrained Stochastic Optimization
by: O'Neill, Michael J.
Published: (2024)

Penalty Interior-Point Method Fails to Converge
by: Leyffer, Sven
Published: (2003)

Randomized Submanifold Subgradient Method for Optimization over Stiefel Manifolds
by: Cheung, Andy Yat-Ming, et al.
Published: (2024)

Efficient Implementation of Third-Order Tensor Methods with Adaptive Regularization for Unconstrained Optimization
by: Cartis, Coralia, et al.
Published: (2024)

Parallel block coordinate descent methods with identification strategies
by: Lopes, Ronaldo, et al.
Published: (2025)

Complexity of an inexact stochastic SQP algorithm for equality constrained optimization
by: O'Neill, Michael J., et al.
Published: (2026)

Inexact Proximal Point Algorithms for Zeroth-Order Global Optimization
by: Zhang, Minxin, et al.
Published: (2024)

A Second-Order Algorithm Based on Affine Scaling Interior-Point Methods for nonlinear Optimisation with bound constraints
by: Pei, Yonggang, et al.
Published: (2026)