Image de couverture de livre

Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Cai, Yuhang, Wu, Jingfeng, Mei, Song, Lindsey, Michael, Bartlett, Peter L.
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Optimization and Control
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Implicit Bias of Gradient Descent for Non-Homogeneous Deep Networks
par: Cai, Yuhang, et autres
Publié: (2025)

Composing Optimized Stepsize Schedules for Gradient Descent
par: Grimmer, Benjamin, et autres
Publié: (2024)

Det-CGD: Compressed Gradient Descent with Matrix Stepsizes for Non-Convex Optimization
par: Li, Hanmin, et autres
Publié: (2023)

Accelerated Gradient Descent by Concatenation of Stepsize Schedules
par: Zhang, Zehao, et autres
Publié: (2024)

A Strengthened Conjecture on the Minimax Optimal Constant Stepsize for Gradient Descent
par: Grimmer, Benjamin, et autres
Publié: (2024)

Accelerating Proximal Gradient Descent via Silver Stepsizes
par: Bok, Jinho, et autres
Publié: (2024)

Stepsize Hedging: an Alternative Mechanism for Accelerating Gradient Descent
par: Altschuler, Jason M., et autres
Publié: (2026)

Non-Euclidean dual gradient ascent for entropically regularized linear and semidefinite programming
par: Cai, Yuhang, et autres
Publié: (2025)

Coupling-based Convergence Diagnostic and Stepsize Scheme for Stochastic Gradient Descent
par: Li, Xiang, et autres
Publié: (2024)

Negative Stepsizes Make Gradient-Descent-Ascent Converge
par: Shugart, Henry, et autres
Publié: (2025)

Large Stepsizes Accelerate Gradient Descent for Regularized Logistic Regression
par: Wu, Jingfeng, et autres
Publié: (2025)

Block Acceleration Without Momentum: On Optimal Stepsizes of Block Gradient Descent for Least-Squares
par: Peng, Liangzu, et autres
Publié: (2024)

Acceleration by Stepsize Hedging I: Multi-Step Descent and the Silver Stepsize Schedule
par: Altschuler, Jason M., et autres
Publié: (2023)

Acceleration by Stepsize Hedging II: Silver Stepsize Schedule for Smooth Convex Optimization
par: Altschuler, Jason M., et autres
Publié: (2023)

Convergence of Implicit Gradient Descent for Training Two-Layer Physics-Informed Neural Networks
par: Xu, Xianliang, et autres
Publié: (2024)

Large Stepsize Gradient Descent for Logistic Loss: Non-Monotonicity of the Loss Improves Optimization Efficiency
par: Wu, Jingfeng, et autres
Publié: (2024)

MARINA-P: Superior Performance in Non-smooth Federated Optimization with Adaptive Stepsizes
par: Sokolov, Igor, et autres
Publié: (2024)

Convergence Analysis of Noisy Distributed Gradient Descent for Non-convex Optimization -- Saddle Point Escape
par: Qin, Lei, et autres
Publié: (2025)

Two-Timescale Gradient Descent Ascent Algorithms for Nonconvex Minimax Optimization
par: Lin, Tianyi, et autres
Publié: (2024)

An Adaptive Stochastic Gradient Method with Non-negative Gauss-Newton Stepsizes
par: Orvieto, Antonio, et autres
Publié: (2024)

Variance Reduced Distributed Non-Convex Optimization Using Matrix Stepsizes
par: Li, Hanmin, et autres
Publié: (2023)

Structured Gradient Descent for Fast Robust Low-Rank Hankel Matrix Completion
par: Cai, HanQin, et autres
Publié: (2022)

A Homogeneous Second-Order Descent Method for Nonconvex Optimization
par: Zhang, Chuwen, et autres
Publié: (2022)

Projected Gradient Descent for Constrained Decision-Dependent Optimization
par: Wang, Zifan, et autres
Publié: (2025)

Last-Iterate Convergence of Anchored Gradient Descent
par: Cai, Yang, et autres
Publié: (2026)

Generalization of Silver Stepsize Schedule to Stochastic Optimization
par: Bai, Luwei, et autres
Publié: (2025)

Adaptive Stepsize Selection in Decentralized Convex Optimization
par: Kuruzov, Ilya, et autres
Publié: (2025)

Time-Reversed BSDEs for Accurate Gradient Estimation in Diffusion Models
par: Mei, Yuhang, et autres
Publié: (2026)

Homogeneous Second-Order Descent Framework: A Fast Alternative to Newton-Type Methods
par: He, Chang, et autres
Publié: (2023)

Gradient Descent for Convex and Smooth Noisy Optimization
par: Hu, Feifei, et autres
Publié: (2024)

AdGT: Decentralized Gradient Tracking with Tuning-free Per-Agent Stepsize
par: Ghaderyan, Diyako, et autres
Publié: (2025)

Non-Euclidean Gradient Descent Operates at the Edge of Stability
par: Islamov, Rustem, et autres
Publié: (2026)

Adaptive Accelerated Gradient Descent Methods for Convex Optimization
par: Xu, Zeyi, et autres
Publié: (2026)

Decentralized Relaxed Smooth Optimization with Gradient Descent Methods
par: Jiang, Zhanhong, et autres
Publié: (2025)

GLinSAT: The General Linear Satisfiability Neural Network Layer By Accelerated Gradient Descent
par: Zeng, Hongtai, et autres
Publié: (2024)

Convergence of Spectral Descent for Non-smooth Optimization
par: Yang, Yixuan, et autres
Publié: (2026)

Adaptive Polyak Stepsize with Level-value Adjustment for Distributed Optimization
par: Ouyang, Chen, et autres
Publié: (2026)

Local Curvature Descent: Squeezing More Curvature out of Standard and Polyak Gradient Descent
par: Richtárik, Peter, et autres
Publié: (2024)

Distributed Optimization and Learning for Automated Stepsize Selection with Finite Time Coordination
par: Rikos, Apostolos I., et autres
Publié: (2025)

Generalized Stochastic Gradient Descent with Momentum Methods for Smooth Optimization
par: Wang, Zimeng, et autres
Publié: (2026)