Imagen de Portada

Regularized Gradient Clipping Provably Trains Wide and Deep Neural Networks

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Tucat, Matteo, Mukherjee, Anirbit, Sen, Procheta, Sun, Mingfei, Rivasplata, Omar
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Optimization and Control
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Global Convergence of SGD On Two Layer Neural Nets
por: Gopalani, Pulkit, et al.
Publicado: (2022)

Global Convergence of SGD For Logistic Loss on Two Layer Neural Nets
por: Gopalani, Pulkit, et al.
Publicado: (2023)

Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse
por: Jacot, Arthur, et al.
Publicado: (2024)

Provable Acceleration of Nesterov's Accelerated Gradient for Rectangular Matrix Factorization and Linear Neural Networks
por: Xu, Zhenghao, et al.
Publicado: (2024)

Provable Accelerated Convergence of Nesterov's Momentum for Deep ReLU Neural Networks
por: Liao, Fangshuo, et al.
Publicado: (2023)

From Gradient Clipping to Normalization for Heavy Tailed SGD
por: Hübler, Florian, et al.
Publicado: (2024)

Parameter-free Clipped Gradient Descent Meets Polyak
por: Takezawa, Yuki, et al.
Publicado: (2024)

SMiLE: Provably Enforcing Global Relational Properties in Neural Networks
por: Francobaldi, Matteo, et al.
Publicado: (2025)

Revisiting Gradient Normalization and Clipping for Nonconvex SGD under Heavy-Tailed Noise: Necessity, Sufficiency, and Acceleration
por: Sun, Tao, et al.
Publicado: (2024)

Provable Acceleration of Nesterov's Accelerated Gradient Method over Heavy Ball Method in Training Over-Parameterized Neural Networks
por: Liu, Xin, et al.
Publicado: (2022)

Neural Network Training Techniques Regularize Optimization Trajectory: An Empirical Study
por: Chen, Cheng, et al.
Publicado: (2020)

Robust and Fast Training via Per-Sample Clipping
por: Nobile, Davide, et al.
Publicado: (2026)

Learning Provably Improves the Convergence of Gradient Descent
por: Song, Qingyu, et al.
Publicado: (2025)

Provably-Stable Neural Network-Based Control of Nonlinear Systems
por: Li, Anran, et al.
Publicado: (2025)

Dual Natural Gradient Descent for Scalable Training of Physics-Informed Neural Networks
por: Jnini, Anas, et al.
Publicado: (2025)

Gradient Clipping Beyond Vector Norms: A Spectral Approach for Matrix-Valued Parameters
por: Yukhimchuk, Alexander, et al.
Publicado: (2026)

Active Learning of Deep Neural Networks via Gradient-Free Cutting Planes
por: Zhang, Erica, et al.
Publicado: (2024)

Convergence of Implicit Gradient Descent for Training Two-Layer Physics-Informed Neural Networks
por: Xu, Xianliang, et al.
Publicado: (2024)

Locally Regularized Sparse Graph by Fast Proximal Gradient Descent
por: Sun, Dongfang, et al.
Publicado: (2024)

Towards Simple and Provable Parameter-Free Adaptive Gradient Methods
por: Tao, Yuanzhe, et al.
Publicado: (2024)

Nonconvex Stochastic Optimization under Heavy-Tailed Noises: Optimal Convergence without Gradient Clipping
por: Liu, Zijian, et al.
Publicado: (2024)

Training Infinitely Deep and Wide Transformers
por: Barboni, Raphaël, et al.
Publicado: (2026)

Regularized Adaptive Momentum Dual Averaging with an Efficient Inexact Subproblem Solver for Training Structured Neural Network
por: Huang, Zih-Syuan, et al.
Publicado: (2024)

Differentially Private Clipped-SGD: High-Probability Convergence with Arbitrary Clipping Level
por: Khah, Saleh Vatan, et al.
Publicado: (2025)

Clipped Gradient Methods for Nonsmooth Convex Optimization under Heavy-Tailed Noise: A Refined Analysis
por: Liu, Zijian
Publicado: (2025)

Clip-and-Verify: Linear Constraint-Driven Domain Clipping for Accelerating Neural Network Verification
por: Zhou, Duo, et al.
Publicado: (2025)

SGD with Partial Hessian for Deep Neural Networks Optimization
por: Sun, Ying, et al.
Publicado: (2024)

Armijo Line-search Can Make (Stochastic) Gradient Descent Provably Faster
por: Vaswani, Sharan, et al.
Publicado: (2025)

Regularized Gauss-Newton for Optimizing Overparameterized Neural Networks
por: Adeoye, Adeyemi D., et al.
Publicado: (2024)

An Adaptive and Stability-Promoting Layerwise Training Approach for Sparse Deep Neural Network Architecture
por: Krishnanunni, C G, et al.
Publicado: (2022)

Fast and Provable Tensor-Train Format Tensor Completion via Precondtioned Riemannian Gradient Descent
por: Bian, Fengmiao, et al.
Publicado: (2025)

Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Stochastic Approach
por: Fernando, Heshan, et al.
Publicado: (2022)

Solving Inverse Problems with Deep Linear Neural Networks: Global Convergence Guarantees for Gradient Descent with Weight Decay
por: Laus, Hannah, et al.
Publicado: (2025)

Almost Sure Convergence Analysis of Differentially Private Stochastic Gradient Methods
por: Mukherjee, Amartya, et al.
Publicado: (2025)

Implicit Bias of Gradient Descent for Non-Homogeneous Deep Networks
por: Cai, Yuhang, et al.
Publicado: (2025)

Non-Asymptotic Global Convergence of PPO-Clip
por: Liu, Yin, et al.
Publicado: (2025)

How Neural Networks Learn the Support is an Implicit Regularization Effect of SGD
por: Beneventano, Pierfrancesco, et al.
Publicado: (2024)

Gradient Regularized Newton Boosting Trees with Global Convergence
por: Zozoulenko, Nikita, et al.
Publicado: (2026)

Provably Faster Gradient Descent via Long Steps
por: Grimmer, Benjamin
Publicado: (2023)

Convergence of Gradient Descent for Recurrent Neural Networks: A Nonasymptotic Analysis
por: Cayci, Semih, et al.
Publicado: (2024)