Cover Image

On the numerical reliability of nonsmooth autodiff: a MaxPool case study

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Boustany, Ryan
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Numerical Analysis Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Gauss-Newton Approach for Min-Max Optimization in Generative Adversarial Networks
by: Mishra, Neel, et al.
Published: (2024)

Solving the Offline and Online Min-Max Problem of Non-smooth Submodular-Concave Functions: A Zeroth-Order Approach
by: Farzin, Amir Ali, et al.
Published: (2026)

A single-loop SPIDER-type stochastic subgradient method for expectation-constrained nonconvex nonsmooth optimization
by: Liu, Wei, et al.
Published: (2025)

On the Stability Connection Between Discrete-Time Algorithms and Their Resolution ODEs: Applications to Min-Max Optimisation
by: Farzin, Amir Ali, et al.
Published: (2026)

Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses
by: Dereich, Steffen, et al.
Published: (2024)

Min-Max Optimisation for Nonconvex-Nonconcave Functions Using a Random Zeroth-Order Extragradient Algorithm
by: Farzin, Amir Ali, et al.
Published: (2025)

Frank--Wolfe algorithms for piecewise star-convex functions with a nonsmooth difference-of-convex structure
by: Millán, R. Díaz, et al.
Published: (2023)

Gradient Descent as a Perceptron Algorithm: Understanding Dynamics and Implicit Acceleration
by: Tyurin, Alexander
Published: (2025)

Continuum-marginal optimal transport: a mesh-free kernel method
by: Nakano, Yumiharu
Published: (2026)

End-to-End Mesh Optimization of a Hybrid Deep Learning Black-Box PDE Solver
by: Ma, Shaocong, et al.
Published: (2024)

Suboptimality bounds for trace-bounded SDPs enable a faster and scalable low-rank SDP solver SDPLR+
by: Huang, Yufan, et al.
Published: (2024)

Gradient is All You Need? How Consensus-Based Optimization can be Interpreted as a Stochastic Relaxation of Gradient Descent
by: Riedl, Konstantin, et al.
Published: (2023)

Numerical analysis of a nonsmooth quasilinear elliptic control problem: II. Finite element discretization and error estimates
by: Clason, Christian, et al.
Published: (2023)

Subhomogeneous Deep Equilibrium Models
by: Sittoni, Pietro, et al.
Published: (2024)

Flattened one-bit stochastic gradient descent: compressed distributed optimization with controlled variance
by: Stollenwerk, Alexander, et al.
Published: (2024)

Efficient Trajectory Inference in Wasserstein Space Using Consecutive Averaging
by: Banerjee, Amartya, et al.
Published: (2024)

Anderson Acceleration in Nonsmooth Problems: Local Convergence via Active Manifold Identification
by: Li, Kexin, et al.
Published: (2024)

KANtrol: A Physics-Informed Kolmogorov-Arnold Network Framework for Solving Multi-Dimensional and Fractional Optimal Control Problems
by: Aghaei, Alireza Afzal
Published: (2024)

Real-time optimal control of high-dimensional parametrized systems by deep learning-based reduced order models
by: Tomasetto, Matteo, et al.
Published: (2024)

Cubic regularized subspace Newton for non-convex optimization
by: Zhao, Jim, et al.
Published: (2024)

Towards Quantifying the Preconditioning Effect of Adam
by: Das, Rudrajit, et al.
Published: (2024)

Super Gradient Descent: Global Optimization requires Global Gradient
by: Achour, Seifeddine
Published: (2024)

Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order models
by: Tomasetto, Matteo, et al.
Published: (2024)

Quantitative Convergences of Lie Group Momentum Optimizers
by: Kong, Lingkai, et al.
Published: (2024)

Learning incomplete factorization preconditioners for GMRES
by: Häusner, Paul, et al.
Published: (2024)

A note on continuous-time online learning
by: Ying, Lexing
Published: (2024)

Symmetry & Critical Points
by: Arjevani, Yossi
Published: (2024)

ADMM for Structured Fractional Minimization
by: Yuan, Ganzhao
Published: (2024)

Efficient Algorithms for Regularized Nonnegative Scale-invariant Low-rank Approximation Models
by: Cohen, Jeremy E., et al.
Published: (2024)

Lyapunov Analysis For Monotonically Forward-Backward Accelerated Algorithms
by: Fu, Mingwei, et al.
Published: (2024)

Using Linearized Optimal Transport to Predict the Evolution of Stochastic Particle Systems
by: Karris, Nicholas, et al.
Published: (2024)

Dimensionality Reduction Techniques for Global Bayesian Optimisation
by: Long, Luo, et al.
Published: (2024)

Fast Unconstrained Optimization via Hessian Averaging and Adaptive Gradient Sampling Methods
by: O'Leary-Roseberry, Thomas, et al.
Published: (2024)

Mathematical Opportunities in Digital Twins (MATH-DT)
by: Antil, Harbir
Published: (2024)

Modeling AdaGrad, RMSProp, and Adam with Integro-Differential Equations
by: Heredia, Carlos
Published: (2024)

Learning truly monotone operators with applications to nonlinear inverse problems
by: Belkouchi, Younes, et al.
Published: (2024)

A Natural Primal-Dual Hybrid Gradient Method for Adversarial Neural Network Training on Solving Partial Differential Equations
by: Liu, Shu, et al.
Published: (2024)

Multi-level Optimal Control with Neural Surrogate Models
by: Kalise, Dante, et al.
Published: (2024)

Riemannian Preconditioned LoRA for Fine-Tuning Foundation Models
by: Zhang, Fangzhao, et al.
Published: (2024)

Nonlinear Assimilation via Score-based Sequential Langevin Sampling
by: Ding, Zhao, et al.
Published: (2024)