Copertina

Non-convergence of Adam and other adaptive stochastic gradient descent optimization methods for non-vanishing learning rates

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Dereich, Steffen, Graeber, Robin, Jentzen, Arnulf
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Optimization and Control Probability 60J22 (Primary), 65K10, 60J20, 65C40 (Secondary) G.1.6; F.2.0; G.3
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Asymptotic stability properties and a priori bounds for Adam and other gradient descent optimization methods
di: Dereich, Steffen, et al.
Pubblicazione: (2025)

Non-convergence to the optimal risk for Adam and stochastic gradient descent optimization in the training of deep neural networks
di: Do, Thang, et al.
Pubblicazione: (2025)

Uniform a priori bounds and error analysis for the Adam stochastic gradient descent optimization method
di: Dereich, Steffen, et al.
Pubblicazione: (2026)

Non-convergence to global minimizers in data driven supervised deep learning: Adam and stochastic gradient descent optimization provably fail to converge to global minimizers in the training of deep neural networks with ReLU activation
di: Do, Thang, et al.
Pubblicazione: (2024)

Convergence of stochastic gradient descent schemes for Lojasiewicz-landscapes
di: Dereich, Steffen, et al.
Pubblicazione: (2021)

Adam symmetry theorem: characterization of the convergence of the stochastic Adam optimizer
di: Dereich, Steffen, et al.
Pubblicazione: (2025)

Extending the noise of splitting to its completion and stability of Brownian maxima
di: Vidmar, Matija, et al.
Pubblicazione: (2024)

Controlled fields, rough stochastic calculus, and Itô-Wentzell-Alekseev-Gröbner identities
di: Dause, Jannis R., et al.
Pubblicazione: (2026)

Uniform in time convergence of numerical schemes for stochastic differential equations via Strong Exponential stability: Euler methods, Split-Step and Tamed Schemes
di: Angeli, Letizia, et al.
Pubblicazione: (2023)

Hedging in Jump Diffusion Model with Transaction Costs
di: Almani, Hamidreza Maleki, et al.
Pubblicazione: (2024)

Cover times with stochastic resetting
di: Linn, Samantha, et al.
Pubblicazione: (2024)

Fast reliable pricing and calibration of the rough Heston model
di: Boyarchenko, Svetlana, et al.
Pubblicazione: (2025)

Optimal convergence rates in stochastic homogenization in a balanced random environment
di: Guo, Xiaoqin, et al.
Pubblicazione: (2023)

Risk-Neutral Pricing of Random-Expiry Options Using Trinomial Trees
di: Bossu, Sebastien, et al.
Pubblicazione: (2025)

Moment estimates, exponential integrability, concentration inequalities and exit times estimates on evolving manifolds
di: Baumgarth, Robert
Pubblicazione: (2024)

Correct implied volatility shapes and reliable pricing in the rough Heston model
di: Boyarchenko, Svetlana, et al.
Pubblicazione: (2024)

Sharp barrier estimates for Bessel bridges
di: Chiarini, Leandro, et al.
Pubblicazione: (2025)

Drawdowns of diffusions
di: Salminen, Paavo, et al.
Pubblicazione: (2024)

Extreme ATM skew in a local volatility model with discontinuity: joint density approach
di: Gairat, Alexander, et al.
Pubblicazione: (2023)

An $L_{q}(L_{p})$-regularity theory for parabolic equations with integro-differential operators having low intensity kernels
di: Kang, Jaehoon, et al.
Pubblicazione: (2023)

A characterisation of the continuum Gaussian free field in $d \geq 2$ dimensions
di: Aru, Juhan, et al.
Pubblicazione: (2021)

Uniform convergence of Dyson Ferrari--Spohn diffusions to the Airy line ensemble
di: Dimitrov, Evgeni, et al.
Pubblicazione: (2023)

Non-asymptotic uniform in time error bounds for new and old numerical schemes for SPDEs
di: Huang, Can, et al.
Pubblicazione: (2026)

The one-shot problem: Solution to an open question of finite-fuel singular control with discretionary stopping
di: Moriarty, John, et al.
Pubblicazione: (2024)

Discretization of the Ergodic Functional Central Limit Theorem
di: Pagès, Gilles, et al.
Pubblicazione: (2018)

Hitting time for one class of Gaussian proccesses
di: Wang, Qingsong, et al.
Pubblicazione: (2024)

On the exponential integrability of the derivative of intersection and self-intersection local time for Brownian motion and related processes
di: Das, Kaustav, et al.
Pubblicazione: (2024)

Asymptotics of lowlying Dirichlet eigenvalues of Witten Laplacians on domains in pinned path groups
di: Aida, Shigeki
Pubblicazione: (2025)

Stochastic Domination of Exit Times for Random Walks and Brownian Motion with Drift
di: Geng, Xi, et al.
Pubblicazione: (2024)

Analysis of an exponential integrator for stochastic PDEs driven by Riesz noise
di: Bréhier, Charles-Edouard, et al.
Pubblicazione: (2026)

Hybrid metapopulation agent-based epidemiological models for efficient insight on the individual scale: a contribution to green computing
di: Bicker, Julia, et al.
Pubblicazione: (2024)

Optimal convergence rates of MCMC integration for functions with unbounded second moment
di: Hofstadler, Julian
Pubblicazione: (2024)

A numerical method to simulate the stochastic linear-quadratic optimal control problem with control constraint in higher dimensions
di: Chaudhary, Abhishek
Pubblicazione: (2024)

Accelerated Simulation Algorithms for Extreme First-Passage Problems with General Emission Profiles
di: Mfoumou, Emmanuel Akame, et al.
Pubblicazione: (2026)

Rough volatility, path-dependent PDEs and weak rates of convergence
di: Bonesini, Ofelia, et al.
Pubblicazione: (2023)

A Theory of Non-Acyclic Generative Flow Networks
di: Brunswic, Leo Maxime, et al.
Pubblicazione: (2023)

Convergence analysis of Tikhonov regularization for non-linear statistical inverse learning problems
di: Rastogi, Abhishake, et al.
Pubblicazione: (2019)

Particle method for the numerical simulation of the path-dependent McKean-Vlasov equation
di: Bernou, Armand, et al.
Pubblicazione: (2022)

Sharp higher order convergence rates for the Adam optimizer
di: Dereich, Steffen, et al.
Pubblicazione: (2025)

Wasserstein error estimates between telegraph processes and Brownian motion
di: Barrera, Gerardo, et al.
Pubblicazione: (2025)