Cover Image

Adaptive Momentum and Nonlinear Damping for Neural Network Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Karoni, Aikaterini, Rajpal, Rajit, Leimkuhler, Benedict, Stoltz, Gabriel
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Adaptive Stepsizing for Stochastic Gradient Langevin Dynamics in Bayesian Neural Networks
by: Rajpal, Rajit, et al.
Published: (2025)

Neighbor-Sampling Based Momentum Stochastic Methods for Training Graph Neural Networks
by: Noel, Molly, et al.
Published: (2025)

Regularized Adaptive Momentum Dual Averaging with an Efficient Inexact Subproblem Solver for Training Structured Neural Network
by: Huang, Zih-Syuan, et al.
Published: (2024)

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks
by: Papazov, Hristo, et al.
Published: (2024)

Provable Accelerated Convergence of Nesterov's Momentum for Deep ReLU Neural Networks
by: Liao, Fangshuo, et al.
Published: (2023)

An Adaptive and Stability-Promoting Layerwise Training Approach for Sparse Deep Neural Network Architecture
by: Krishnanunni, C G, et al.
Published: (2022)

Adaptive Optimization via Momentum on Variance-Normalized Gradients
by: Patitucci, Francisco, et al.
Published: (2026)

SGD with Adaptive Preconditioning: Unified Analysis and Momentum Acceleration
by: Kovalev, Dmitry
Published: (2025)

Keep the Momentum: Conservation Laws beyond Euclidean Gradient Flows
by: Marcotte, Sibylle, et al.
Published: (2024)

Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models
by: Xie, Xingyu, et al.
Published: (2022)

Provably-Stable Neural Network-Based Control of Nonlinear Systems
by: Li, Anran, et al.
Published: (2025)

Generative modeling of conditional probability distributions on the level-sets of collective variables
by: Akhyar, Fatima-Zahrae, et al.
Published: (2025)

Relaxation-Informed Training of Neural Network Surrogate Models
by: Tsay, Calvin
Published: (2026)

Evaluating probabilistic and data-driven inference models for fiber-coupled NV-diamond temperature sensors
by: Rajpal, Shraddha, et al.
Published: (2024)

Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse
by: Jacot, Arthur, et al.
Published: (2024)

A Guaranteed-Stable Neural Network Approach for Optimal Control of Nonlinear Systems
by: Li, Anran, et al.
Published: (2025)

Compression-aware Training of Neural Networks using Frank-Wolfe
by: Zimmer, Max, et al.
Published: (2022)

Universal Approximation Power of Deep Residual Neural Networks via Nonlinear Control Theory
by: Tabuada, Paulo, et al.
Published: (2020)

Optimization Over Trained Neural Networks: Taking a Relaxing Walk
by: Tong, Jiatai, et al.
Published: (2024)

Towards Guided Descent: Optimization Algorithms for Training Neural Networks At Scale
by: Nagwekar, Ansh
Published: (2025)

Regularized Gradient Clipping Provably Trains Wide and Deep Neural Networks
by: Tucat, Matteo, et al.
Published: (2024)

Convex Formulations for Training Two-Layer ReLU Neural Networks
by: Prakhya, Karthik, et al.
Published: (2024)

Neural Network Training Techniques Regularize Optimization Trajectory: An Empirical Study
by: Chen, Cheng, et al.
Published: (2020)

A Nonlinear Separation Principle via Contraction Theory: Applications to Neural Networks, Control, and Learning
by: Gokhale, Anand, et al.
Published: (2026)

Discrete-time Contraction-based Control of Nonlinear Systems with Parametric Uncertainties using Neural Networks
by: Wei, Lai, et al.
Published: (2021)

A Non-Monotone Preconditioned Trust-Region Method for Neural Network Training
by: Angino, Andrea, et al.
Published: (2026)

Dual Natural Gradient Descent for Scalable Training of Physics-Informed Neural Networks
by: Jnini, Anas, et al.
Published: (2025)

A Framework for Adaptive Stabilisation of Nonlinear Stochastic Systems
by: Siriya, Seth, et al.
Published: (2025)

Parameter-Adaptive Approximate MPC: Tuning Neural-Network Controllers without Retraining
by: Hose, Henrik, et al.
Published: (2024)

Contraction-Guided Adaptive Partitioning for Reachability Analysis of Neural Network Controlled Systems
by: Harapanahalli, Akash, et al.
Published: (2023)

Why Line Search when you can Plane Search? SO-Friendly Neural Networks allow Per-Iteration Optimization of Learning and Momentum Rates for Every Layer
by: Shea, Betty, et al.
Published: (2024)

Fourier Learning Machines: Nonharmonic Fourier-Based Neural Networks for Scientific Machine Learning
by: Rubel, Mominul, et al.
Published: (2025)

Mean-Field Limits for Two-Layer Neural Networks Trained with Consensus-Based Optimization
by: De Deyn, William, et al.
Published: (2025)

Multi-Objective Linear Ensembles for Robust and Sparse Training of Few-Bit Neural Networks
by: Bernardelli, Ambrogio Maria, et al.
Published: (2022)

Enhancing Stability of Physics-Informed Neural Network Training Through Saddle-Point Reformulation
by: Bylinkin, Dmitry, et al.
Published: (2025)

Convergence of Implicit Gradient Descent for Training Two-Layer Physics-Informed Neural Networks
by: Xu, Xianliang, et al.
Published: (2024)

Optimization over Trained (and Sparse) Neural Networks: A Surrogate within a Surrogate
by: Pham, Hung, et al.
Published: (2025)

Stochastic Difference-of-Convex Optimization with Momentum
by: Chayti, El Mahdi, et al.
Published: (2025)

Improving Stochastic Cubic Newton with Momentum
by: Chayti, El Mahdi, et al.
Published: (2024)

Dimension-adapted Momentum Outscales SGD
by: Ferbach, Damien, et al.
Published: (2025)