Cover Image

Averaged Adam accelerates stochastic optimization in the training of deep neural network approximations for partial differential equation and optimal control problems

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Dereich, Steffen, Jentzen, Arnulf, Riekert, Adrian
Format:	Preprint
Published:	2025
Subjects:	Optimization and Control Machine Learning Numerical Analysis
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses
by: Dereich, Steffen, et al.
Published: (2024)

PADAM: Parallel averaged Adam reduces the error for stochastic optimization in scientific machine learning
by: Jentzen, Arnulf, et al.
Published: (2025)

Non-convergence to global minimizers for Adam and stochastic gradient descent optimization and constructions of local minimizers in the training of artificial neural networks
by: Jentzen, Arnulf, et al.
Published: (2024)

On the existence of minimizers in shallow residual ReLU neural network optimization landscapes
by: Dereich, Steffen, et al.
Published: (2023)

Sharp higher order convergence rates for the Adam optimizer
by: Dereich, Steffen, et al.
Published: (2025)

Convergence rates for the Adam optimizer
by: Dereich, Steffen, et al.
Published: (2024)

Non-convergence to the optimal risk for Adam and stochastic gradient descent optimization in the training of deep neural networks
by: Do, Thang, et al.
Published: (2025)

Asymptotic stability properties and a priori bounds for Adam and other gradient descent optimization methods
by: Dereich, Steffen, et al.
Published: (2025)

ODE approximation for the Adam algorithm: General and overparametrized setting
by: Dereich, Steffen, et al.
Published: (2025)

Martingale deep learning for very high dimensional quasi-linear partial differential equations and stochastic optimal controls
by: Cai, Wei, et al.
Published: (2024)

Space-time deep neural network approximations for high-dimensional partial differential equations
by: Hornung, Fabian, et al.
Published: (2020)

Algorithmically Designed Artificial Neural Networks (ADANNs): Higher order deep operator learning for parametric partial differential equations
by: Jentzen, Arnulf, et al.
Published: (2023)

Non-convergence to global minimizers in data driven supervised deep learning: Adam and stochastic gradient descent optimization provably fail to converge to global minimizers in the training of deep neural networks with ReLU activation
by: Do, Thang, et al.
Published: (2024)

Uniform a priori bounds and error analysis for the Adam stochastic gradient descent optimization method
by: Dereich, Steffen, et al.
Published: (2026)

Non-convergence of Adam and other adaptive stochastic gradient descent optimization methods for non-vanishing learning rates
by: Dereich, Steffen, et al.
Published: (2024)

Iterative solvers for partial differential equations with dissipative structure: Operator preconditioning and optimal control
by: Mehrmann, Volker, et al.
Published: (2025)

Two-scale neural networks for optimal control of linear convection-dominated equations
by: Liu, Sijing, et al.
Published: (2026)

Adam symmetry theorem: characterization of the convergence of the stochastic Adam optimizer
by: Dereich, Steffen, et al.
Published: (2025)

Convergence to good non-optimal critical points in the training of neural networks: Gradient descent optimization with one random initialization overcomes all bad non-global local minima with high probability
by: Ibragimov, Shokhrukh, et al.
Published: (2022)

High-dimensional approximation spaces of artificial neural networks and applications to partial differential equations
by: Beneventano, Pierfrancesco, et al.
Published: (2020)

On the existence of optimal shallow feedforward networks with ReLU activation
by: Dereich, Steffen, et al.
Published: (2023)

Contractivity of neural ODEs: an eigenvalue optimization problem
by: Guglielmi, Nicola, et al.
Published: (2024)

Ordinary differential equations for regularized variational problems involving semi-discrete optimal transport
by: Cances, Adrien, et al.
Published: (2026)

Convergence analysis for an implementable scheme to solve the linear-quadratic stochastic optimal control problem with stochastic wave equation
by: Chaudhary, Abhishek
Published: (2025)

An accelerated gradient method with adaptive restart for convex multiobjective optimization problems
by: Luo, Hao, et al.
Published: (2025)

Numerical approximations for partially observed optimal control of stochastic partial differential equations
by: Bao, Feng, et al.
Published: (2025)

A finite element scheme for an optimal control problem on steady Navier-Stokes-Brinkman equations
by: Araneda, Jorge Aguayo, et al.
Published: (2025)

An approximate Itô-SDE based simulated annealing algorithm for multivariate design optimization problems
by: Batou, A.
Published: (2019)

Unidimensional semi-discrete partial optimal transport
by: Cances, Adrien, et al.
Published: (2025)

Modelling sand ripples in mine countermeasure simulations by means of stochastic optimal control
by: Blondeel, Philippe, et al.
Published: (2024)

Volume-preserving geometric shape optimization of the Dirichlet energy using variational neural networks
by: Bélières--Frendo, Amaury, et al.
Published: (2024)

Error analysis for stochastic gradient optimization schemes using modified equations
by: Bréhier, Charles-Edouard, et al.
Published: (2024)

Non-overlapping Schwarz methods in time for parabolic optimal control problems
by: Gander, Martin Jakob, et al.
Published: (2024)

Characterizing and computing solutions to regularized semi-discrete optimal transport via an ordinary differential equation
by: Nenna, Luca, et al.
Published: (2025)

Anderson-type acceleration method for Deep Neural Network optimization
by: Ito, Kazufumi, et al.
Published: (2025)

Deep learning based numerical approximation algorithms for stochastic partial differential equations
by: Beck, Christian, et al.
Published: (2020)

H2 optimal rational approximation on general domains
by: Borghi, Alessandro, et al.
Published: (2023)

An augmented Lagrangian trust-region method with inexact gradient evaluations to accelerate constrained optimization problems using model hyperreduction
by: Wen, Tianshu, et al.
Published: (2024)

Numerical solution of elliptic distributed optimal control problems with boundary value tracking
by: Langer, Ulrich, et al.
Published: (2025)

From PDEs constrained optimization to controllability problems via time domain decomposition
by: Cocquet, Pierre-Henri, et al.
Published: (2026)