Cover Image

Non-asymptotic convergence analysis of the stochastic gradient Hamiltonian Monte Carlo algorithm with discontinuous stochastic gradient with applications to training of ReLU neural networks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liang, Luxu, Neufeld, Ariel, Zhang, Ying
Format:	Preprint
Published:	2024
Subjects:	Optimization and Control Machine Learning Numerical Analysis Probability
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Langevin dynamics based algorithm e-TH$\varepsilon$O POULA for stochastic optimization problems with discontinuous stochastic gradient
by: Lim, Dong-Young, et al.
Published: (2022)

Provably convergent stochastic fixed-point algorithm for free-support Wasserstein barycenter of continuous non-parametric measures
by: Chen, Zeyi, et al.
Published: (2025)

Non-asymptotic convergence bounds for modified tamed unadjusted Langevin algorithm in non-convex setting
by: Neufeld, Ariel, et al.
Published: (2022)

Multilevel Picard approximations and deep neural networks with ReLU, leaky ReLU, and softplus activation overcome the curse of dimensionality when approximating semilinear parabolic partial differential equations in $L^p$-sense
by: Neufeld, Ariel, et al.
Published: (2024)

Deep ReLU neural networks overcome the curse of dimensionality when approximating semilinear partial integro-differential equations
by: Neufeld, Ariel, et al.
Published: (2023)

Non-convergence to global minimizers in data driven supervised deep learning: Adam and stochastic gradient descent optimization provably fail to converge to global minimizers in the training of deep neural networks with ReLU activation
by: Do, Thang, et al.
Published: (2024)

Error analysis for stochastic gradient optimization schemes using modified equations
by: Bréhier, Charles-Edouard, et al.
Published: (2024)

Rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of gradient-dependent semilinear heat equations
by: Neufeld, Ariel, et al.
Published: (2024)

Solving stochastic partial differential equations using neural networks in the Wiener chaos expansion
by: Neufeld, Ariel, et al.
Published: (2024)

Rectified deep neural networks overcome the curse of dimensionality when approximating solutions of McKean--Vlasov stochastic differential equations
by: Neufeld, Ariel, et al.
Published: (2023)

Multilevel Picard algorithm for general semilinear parabolic PDEs with gradient-dependent nonlinearities
by: Neufeld, Ariel, et al.
Published: (2023)

Numerical method for approximately optimal solutions of two-stage distributionally robust optimization with marginal constraints
by: Neufeld, Ariel, et al.
Published: (2022)

Numerical method for feasible and approximately optimal solutions of multi-marginal optimal transport beyond discrete measures
by: Neufeld, Ariel, et al.
Published: (2022)

Feasible approximation of matching equilibria for large-scale matching for teams problems
by: Neufeld, Ariel, et al.
Published: (2023)

On the convergence of stochastic variance reduced gradient for linear inverse problems
by: Jin, Bangti, et al.
Published: (2025)

Numerical method for nonlinear Kolmogorov PDEs via sensitivity analysis
by: Bartl, Daniel, et al.
Published: (2024)

Non-concave stochastic optimal control in finite discrete time under model uncertainty
by: Neufeld, Ariel, et al.
Published: (2024)

A stochastic gradient descent algorithm with random search directions
by: Gbaguidi, Eméric
Published: (2025)

Deep learning based numerical approximation algorithms for stochastic partial differential equations
by: Beck, Christian, et al.
Published: (2020)

Non-asymptotic estimates for accelerated high order Langevin Monte Carlo algorithms
by: Neufeld, Ariel, et al.
Published: (2024)

Multilevel Picard approximations for McKean-Vlasov stochastic differential equations with nonconstant diffusion
by: Neufeld, Ariel, et al.
Published: (2025)

Convergence analysis for an implementable scheme to solve the linear-quadratic stochastic optimal control problem with stochastic wave equation
by: Chaudhary, Abhishek
Published: (2025)

Numerical approximation of McKean-Vlasov SDEs via stochastic gradient descent
by: Agarwal, Ankush, et al.
Published: (2023)

Non-exchangeable evolutionary and mean field games and their applications
by: Yoshioka, H., et al.
Published: (2025)

Randomized coordinate gradient descent almost surely escapes strict saddle points
by: Chen, Ziang, et al.
Published: (2025)

Multilevel Picard approximation algorithm for semilinear partial integro-differential equations and its complexity analysis
by: Neufeld, Ariel, et al.
Published: (2022)

Multilevel Picard approximations overcome the curse of dimensionality in the numerical approximation of general semilinear PDEs with gradient-dependent nonlinearities
by: Neufeld, Ariel, et al.
Published: (2023)

An abstract effective convergence theorem for stochastic processes, with applications to stochastic approximation
by: Neri, Morenikeji, et al.
Published: (2025)

Convergence rates for gradient descent in the training of overparameterized artificial neural networks with piecewise affine activation
by: Jentzen, Arnulf, et al.
Published: (2021)

Markov decision processes: on the convergence of the Monte-Carlo first visit algorithm
by: Delattre, Sylvain, et al.
Published: (2025)

On the existence of minimizers in shallow residual ReLU neural network optimization landscapes
by: Dereich, Steffen, et al.
Published: (2023)

On the convergence of adaptive approximations for stochastic differential equations
by: Foster, James, et al.
Published: (2023)

The generator gradient estimator is an adjoint state method for stochastic differential equations
by: Badolle, Quentin, et al.
Published: (2024)

Bounding the Difference between the Values of Robust and Non-Robust Markov Decision Problems
by: Neufeld, Ariel, et al.
Published: (2023)

Stably unactivated neurons in ReLU neural networks
by: Brownlowe, Natalie, et al.
Published: (2024)

Robust SGLD algorithm for solving non-convex distributionally robust optimisation problems
by: Neufeld, Ariel, et al.
Published: (2024)

Deep neural networks with ReLU, leaky ReLU, and softplus activation provably overcome the curse of dimensionality for space-time solutions of semilinear partial differential equations
by: Ackermann, Julia, et al.
Published: (2024)

Deep learning algorithms for FBSDEs with jumps: Applications to option pricing and a MFG model for smart grids
by: Alasseur, Clémence, et al.
Published: (2024)

On the existence of optimal shallow feedforward networks with ReLU activation
by: Dereich, Steffen, et al.
Published: (2023)

Fast Spawn\&Prune (FS\&P): Global convergence of stochastic conic particle gradient descent via birth/death process
by: De Castro, Yohann, et al.
Published: (2026)