Cover Image

The duality structure gradient descent algorithm: analysis and applications to neural networks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Flynn, Thomas
Format:	Preprint
Published:	2017
Subjects:	Machine Learning Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Convergence of continuous-time stochastic gradient descent with applications to deep neural networks
by: Lugosi, Gabor, et al.
Published: (2024)

Convergence of stochastic gradient descent under a local Lojasiewicz condition for deep neural networks
by: An, Jing, et al.
Published: (2023)

A stochastic gradient descent algorithm with random search directions
by: Gbaguidi, Eméric
Published: (2025)

Non-convergence to global minimizers for Adam and stochastic gradient descent optimization and constructions of local minimizers in the training of artificial neural networks
by: Jentzen, Arnulf, et al.
Published: (2024)

Problem-dependent convergence bounds for randomized linear gradient compression
by: Flynn, Thomas, et al.
Published: (2024)

Non-asymptotic convergence analysis of the stochastic gradient Hamiltonian Monte Carlo algorithm with discontinuous stochastic gradient with applications to training of ReLU neural networks
by: Liang, Luxu, et al.
Published: (2024)

Precise gradient descent training dynamics for finite-width multi-layer neural networks
by: Han, Qiyang, et al.
Published: (2025)

New logarithmic step size for stochastic gradient descent
by: Shamaee, M. Soheil, et al.
Published: (2024)

Convergence of gradient flow for learning convolutional neural networks
by: Diederen, Jona-Maria, et al.
Published: (2026)

Riemannian coordinate descent algorithms on matrix manifolds
by: Han, Andi, et al.
Published: (2024)

On the global convergence of gradient descent for wide shallow models with bounded nonlinearities
by: Petit, Romain, et al.
Published: (2026)

On the stability of gradient descent with second order dynamics for time-varying cost functions
by: Gibson, Travis E., et al.
Published: (2024)

Convergence of gradient descent for deep neural networks
by: Chatterjee, Sourav
Published: (2022)

Error dynamics of mini-batch gradient descent with random reshuffling for least squares regression
by: Lok, Jackie, et al.
Published: (2024)

The late-stage training dynamics of (stochastic) subgradient descent on homogeneous neural networks
by: Schechtman, Sholom, et al.
Published: (2025)

Global convergence of gradient descent for phase retrieval
by: Fougereux, Théodore, et al.
Published: (2024)

Recurrent neural networks: vanishing and exploding gradients are not the end of the story
by: Zucchet, Nicolas, et al.
Published: (2024)

A short proof of near-linear convergence of adaptive gradient descent under fourth-order growth and convexity
by: Davis, Damek, et al.
Published: (2026)

An accelerated first-order regularized momentum descent ascent algorithm for stochastic nonconvex-concave minimax problems
by: Zhang, Huiling, et al.
Published: (2023)

An analysis of optimization problems involving ReLU neural networks
by: Plate, Christoph, et al.
Published: (2025)

When majority rules, minority loses: bias amplification of gradient descent
by: Bachoc, François, et al.
Published: (2025)

Graph neural networks for the prediction of molecular structure-property relationships
by: Rittig, Jan G., et al.
Published: (2022)

Polygonal Unadjusted Langevin Algorithms: Creating stable and efficient adaptive algorithms for neural networks
by: Lim, Dong-Young, et al.
Published: (2021)

Manifold constrained steepest descent
by: Yang, Kaiwei, et al.
Published: (2026)

Flattened one-bit stochastic gradient descent: compressed distributed optimization with controlled variance
by: Stollenwerk, Alexander, et al.
Published: (2024)

Convergence of flow-based generative models via proximal gradient descent in Wasserstein space
by: Cheng, Xiuyuan, et al.
Published: (2023)

Long-time dynamics and universality of nonconvex gradient descent
by: Han, Qiyang
Published: (2025)

Adaptive control mechanisms in gradient descent algorithms
by: Iannelli, Andrea
Published: (2025)

Convergence of two-timescale gradient descent ascent dynamics: finite-dimensional and mean-field perspectives
by: An, Jing, et al.
Published: (2025)

Learning mirror maps in policy mirror descent
by: Alfano, Carlo, et al.
Published: (2024)

Mean-field neural networks-based algorithms for McKean-Vlasov control problems *
by: Pham, Huyên, et al.
Published: (2022)

Error bounds for particle gradient descent, and extensions of the log-Sobolev and Talagrand inequalities
by: Caprio, Rocco, et al.
Published: (2024)

Full error analysis of policy gradient learning algorithms for exploratory linear quadratic mean-field control problem in continuous time with common noise
by: Frikha, Noufel, et al.
Published: (2024)

Gradient descent in matrix factorization: Understanding large initialization
by: Chen, Hengchao, et al.
Published: (2023)

Size and depth of monotone neural networks: interpolation and approximation
by: Mikulincer, Dan, et al.
Published: (2022)

Quadratic models for understanding catapult dynamics of neural networks
by: Zhu, Libin, et al.
Published: (2022)

Mathematical analysis of one-layer neural network with fixed biases, a new activation function and other observations
by: Macià, Fabricio, et al.
Published: (2026)

Smart energy management: process structure-based hybrid neural networks for optimal scheduling and economic predictive control in integrated systems
by: Wu, Long, et al.
Published: (2024)

Reliably-stabilizing piecewise-affine neural network controllers
by: Fabiani, Filippo, et al.
Published: (2021)

Iterative regularization in classification via hinge loss diagonal descent
by: Apidopoulos, Vassilis, et al.
Published: (2022)