Buchumschlag

Convergence rates for gradient descent in the training of overparameterized artificial neural networks with piecewise affine activation

Fuente: arXiv

Gespeichert in:

Bibliographische Detailangaben
Hauptverfasser:	Jentzen, Arnulf, Kröger, Timo
Format:	Preprint
Veröffentlicht:	2021
Schlagworte:	Machine Learning Numerical Analysis Probability
Online-Zugang:	Acceder al recurso
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Ähnliche Einträge

Space-time deep neural network approximations for high-dimensional partial differential equations
von: Hornung, Fabian, et al.
Veröffentlicht: (2020)

Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses
von: Dereich, Steffen, et al.
Veröffentlicht: (2024)

Deep learning based numerical approximation algorithms for stochastic partial differential equations
von: Beck, Christian, et al.
Veröffentlicht: (2020)

Non-convergence to global minimizers in data driven supervised deep learning: Adam and stochastic gradient descent optimization provably fail to converge to global minimizers in the training of deep neural networks with ReLU activation
von: Do, Thang, et al.
Veröffentlicht: (2024)

Averaged Adam accelerates stochastic optimization in the training of deep neural network approximations for partial differential equation and optimal control problems
von: Dereich, Steffen, et al.
Veröffentlicht: (2025)

Non-convergence to the optimal risk for Adam and stochastic gradient descent optimization in the training of deep neural networks
von: Do, Thang, et al.
Veröffentlicht: (2025)

Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory
von: Jentzen, Arnulf, et al.
Veröffentlicht: (2023)

On bounds for norms of reparameterized ReLU artificial neural network parameters: sums of fractional powers of the Lipschitz norm control the network parameter vector
von: Jentzen, Arnulf, et al.
Veröffentlicht: (2022)

Convergence rates for the Adam optimizer
von: Dereich, Steffen, et al.
Veröffentlicht: (2024)

Non-asymptotic convergence analysis of the stochastic gradient Hamiltonian Monte Carlo algorithm with discontinuous stochastic gradient with applications to training of ReLU neural networks
von: Liang, Luxu, et al.
Veröffentlicht: (2024)

Non-convergence to global minimizers for Adam and stochastic gradient descent optimization and constructions of local minimizers in the training of artificial neural networks
von: Jentzen, Arnulf, et al.
Veröffentlicht: (2024)

Deep neural networks with ReLU, leaky ReLU, and softplus activation provably overcome the curse of dimensionality for space-time solutions of semilinear partial differential equations
von: Ackermann, Julia, et al.
Veröffentlicht: (2024)

Deep neural networks with ReLU, leaky ReLU, and softplus activation provably overcome the curse of dimensionality for Kolmogorov partial differential equations with Lipschitz nonlinearities in the $L^p$-sense
von: Ackermann, Julia, et al.
Veröffentlicht: (2023)

Deep neural networks can provably solve Bellman equations for Markov decision processes without the curse of dimensionality
von: Jentzen, Arnulf, et al.
Veröffentlicht: (2025)

Solving stochastic partial differential equations using neural networks in the Wiener chaos expansion
von: Neufeld, Ariel, et al.
Veröffentlicht: (2024)

Algorithmically Designed Artificial Neural Networks (ADANNs): Higher order deep operator learning for parametric partial differential equations
von: Jentzen, Arnulf, et al.
Veröffentlicht: (2023)

Multilevel Picard approximations and deep neural networks with ReLU, leaky ReLU, and softplus activation overcome the curse of dimensionality when approximating semilinear parabolic partial differential equations in $L^p$-sense
von: Neufeld, Ariel, et al.
Veröffentlicht: (2024)

Fast training of accurate physics-informed neural networks without gradient descent
von: Datar, Chinmay, et al.
Veröffentlicht: (2024)

Mathematical analysis of the gradients in deep learning
von: Dereich, Steffen, et al.
Veröffentlicht: (2025)

High-dimensional approximation spaces of artificial neural networks and applications to partial differential equations
von: Beneventano, Pierfrancesco, et al.
Veröffentlicht: (2020)

Learning the random variables in Monte Carlo simulations with stochastic gradient descent: Machine learning for parametric PDEs and financial derivative pricing
von: Becker, Sebastian, et al.
Veröffentlicht: (2022)

PADAM: Parallel averaged Adam reduces the error for stochastic optimization in scientific machine learning
von: Jentzen, Arnulf, et al.
Veröffentlicht: (2025)

On the existence of minimizers in shallow residual ReLU neural network optimization landscapes
von: Dereich, Steffen, et al.
Veröffentlicht: (2023)

Riemannian Langevin Dynamics: Strong Convergence of Geometric Euler-Maruyama Scheme
von: Zhan, Zhiyuan, et al.
Veröffentlicht: (2026)

Weak convergence rates for temporal numerical approximations of stochastic wave equations with multiplicative noise
von: Cox, Sonja, et al.
Veröffentlicht: (2019)

From Monte Carlo to neural networks approximations of boundary value problems
von: Beznea, Lucian, et al.
Veröffentlicht: (2022)

A brief review of the Deep BSDE method for solving high-dimensional partial differential equations
von: Han, Jiequn, et al.
Veröffentlicht: (2025)

Numerical approximation of McKean-Vlasov SDEs via stochastic gradient descent
von: Agarwal, Ankush, et al.
Veröffentlicht: (2023)

An Overview on Machine Learning Methods for Partial Differential Equations: from Physics Informed Neural Networks to Deep Operator Learning
von: Gonon, Lukas, et al.
Veröffentlicht: (2024)

Convergence of Kinetic Langevin Monte Carlo on Lie groups
von: Kong, Lingkai, et al.
Veröffentlicht: (2024)

Deep neural network approximation theory for high-dimensional functions
von: Beneventano, Pierfrancesco, et al.
Veröffentlicht: (2021)

Rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of gradient-dependent semilinear heat equations
von: Neufeld, Ariel, et al.
Veröffentlicht: (2024)

Strong convergence rates for explicit space-time discrete numerical approximations of stochastic Allen-Cahn equations
von: Becker, Sebastian, et al.
Veröffentlicht: (2017)

Energy stable neural network for gradient flow equations
von: Wu, Yue, et al.
Veröffentlicht: (2023)

Langevin dynamics based algorithm e-TH$\varepsilon$O POULA for stochastic optimization problems with discontinuous stochastic gradient
von: Lim, Dong-Young, et al.
Veröffentlicht: (2022)

Gradient-free training of recurrent neural networks
von: Bolager, Erik Lien, et al.
Veröffentlicht: (2024)

Convergence of two-timescale gradient descent ascent dynamics: finite-dimensional and mean-field perspectives
von: An, Jing, et al.
Veröffentlicht: (2025)

One model to solve them all: 2BSDE families via neural operators
von: Furuya, Takashi, et al.
Veröffentlicht: (2025)

Why Cannot Neural Networks Master Extrapolation? Insights from Physical Laws
von: Dakhmouche, Ramzi, et al.
Veröffentlicht: (2025)

New Trends in the Stability of Sinkhorn Semigroups
von: Del Moral, Pierre, et al.
Veröffentlicht: (2026)