Cover Image

Random Scaling and Momentum for Non-smooth Non-convex Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Qinzi, Cutkosky, Ashok
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversion
by: Cutkosky, Ashok, et al.
Published: (2023)

Reevaluating Theoretical Analysis Methods for Optimization in Deep Learning
by: Tran, Hoang, et al.
Published: (2024)

Private Zeroth-Order Nonsmooth Nonconvex Optimization
by: Zhang, Qinzi, et al.
Published: (2024)

Unconstrained Robust Online Convex Optimization
by: Zhang, Jiujia, et al.
Published: (2025)

Non-convex Stochastic Composite Optimization with Polyak Momentum
by: Gao, Yuan, et al.
Published: (2024)

Adam with model exponential moving average is effective for nonconvex optimization
by: Ahn, Kwangjun, et al.
Published: (2024)

Fully Unconstrained Online Learning
by: Cutkosky, Ashok, et al.
Published: (2024)

Parameter-free Mirror Descent
by: Jacobsen, Andrew, et al.
Published: (2022)

Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance
by: Zhang, Qi, et al.
Published: (2024)

Online Non-convex Optimization with Long-term Non-convex Constraints
by: Pan, Shijie, et al.
Published: (2023)

General framework for online-to-nonconvex conversion: Schedule-free SGD is also effective for nonconvex optimization
by: Ahn, Kwangjun, et al.
Published: (2024)

Convergence of Spectral Descent for Non-smooth Optimization
by: Yang, Yixuan, et al.
Published: (2026)

Decentralized Non-convex Stochastic Optimization with Heterogeneous Variance
by: Chen, Hongxu, et al.
Published: (2026)

A Stochastic Quasi-Newton Method for Non-convex Optimization with Non-uniform Smoothness
by: Sun, Zhenyu, et al.
Published: (2024)

RanSOM: Second-Order Momentum with Randomized Scaling for Constrained and Unconstrained Optimization
by: Chayti, El Mahdi
Published: (2026)

Retraction-Free Decentralized Non-convex Optimization with Orthogonal Constraints
by: Sun, Youbang, et al.
Published: (2024)

MARINA-P: Superior Performance in Non-smooth Federated Optimization with Adaptive Stepsizes
by: Sokolov, Igor, et al.
Published: (2024)

Riemannian Optimization for Non-convex Euclidean Distance Geometry with Global Recovery Guarantees
by: Smith, Chandler, et al.
Published: (2024)

Non-geodesically-convex optimization in the Wasserstein space
by: Luu, Hoang Phuc Hau, et al.
Published: (2024)

Momentum Benefits Non-IID Federated Learning Simply and Provably
by: Cheng, Ziheng, et al.
Published: (2023)

Complexity Lower Bounds of Adaptive Gradient Algorithms for Non-convex Stochastic Optimization under Relaxed Smoothness
by: Crawshaw, Michael, et al.
Published: (2025)

Safeguarded Stochastic Polyak Step Sizes for Non-smooth Optimization: Robust Performance Without Small (Sub)Gradients
by: Oikonomou, Dimitris, et al.
Published: (2025)

SPAM: Stochastic Proximal Point Method with Momentum Variance Reduction for Non-convex Cross-Device Federated Learning
by: Karagulyan, Avetik, et al.
Published: (2024)

Methods with Local Steps and Random Reshuffling for Generally Smooth Non-Convex Federated Optimization
by: Demidovich, Yury, et al.
Published: (2024)

Quantum Algorithms for Non-smooth Non-convex Optimization
by: Liu, Chengchang, et al.
Published: (2024)

Both Asymptotic and Non-Asymptotic Convergence of Quasi-Hyperbolic Momentum using Increasing Batch Size
by: Imaizumi, Kento, et al.
Published: (2025)

Convergence of Adam for Non-convex Objectives: Relaxed Hyperparameters and Non-ergodic Case
by: He, Meixuan, et al.
Published: (2023)

Zeroth-Order Non-smooth Non-convex Optimization via Gaussian Smoothing
by: Paul, Anik Kumar, et al.
Published: (2025)

Stochastic Difference-of-Convex Optimization with Momentum
by: Chayti, El Mahdi, et al.
Published: (2025)

Empirical and computer-aided robustness analysis of long-step and accelerated methods in smooth convex optimization
by: Vernimmen, Pierre, et al.
Published: (2025)

Logarithmic-time Schedules for Scaling Language Models with Momentum
by: Ferbach, Damien, et al.
Published: (2026)

Shuffling Momentum Gradient Algorithm for Convex Optimization
by: Tran, Trang H., et al.
Published: (2024)

Non-Myopic Multifidelity Bayesian Optimization
by: Di Fiore, Francesco, et al.
Published: (2022)

Non-convex entropic mean-field optimization via Best Response flow
by: Lascu, Razvan-Andrei, et al.
Published: (2025)

The Road Less Scheduled
by: Defazio, Aaron, et al.
Published: (2024)

Convex and Non-convex Federated Learning with Stale Stochastic Gradients: Diminishing Step Size is All You Need
by: Zheng, Xinran, et al.
Published: (2026)

Adaptive Optimization via Momentum on Variance-Normalized Gradients
by: Patitucci, Francisco, et al.
Published: (2026)

On the Provable Suboptimality of Momentum SGD in Nonstationary Stochastic Optimization
by: Sahu, Sharan, et al.
Published: (2026)

Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
by: Choudhury, Sayantan, et al.
Published: (2026)

Extended convexity and smoothness and their applications in deep learning
by: Qi, Binchuan, et al.
Published: (2024)