Cover Image

On bounds for norms of reparameterized ReLU artificial neural network parameters: sums of fractional powers of the Lipschitz norm control the network parameter vector

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Jentzen, Arnulf, Kröger, Timo
Format:	Preprint
Published:	2022
Subjects:	Machine Learning Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

On the existence of minimizers in shallow residual ReLU neural network optimization landscapes
by: Dereich, Steffen, et al.
Published: (2023)

MIQCQP reformulation of the ReLU neural networks Lipschitz constant estimation problem
by: Sbihi, Mohammed, et al.
Published: (2024)

Non-convergence to global minimizers for Adam and stochastic gradient descent optimization and constructions of local minimizers in the training of artificial neural networks
by: Jentzen, Arnulf, et al.
Published: (2024)

Gradient descent provably escapes saddle points in the training of shallow ReLU networks
by: Cheridito, Patrick, et al.
Published: (2022)

An analysis of optimization problems involving ReLU neural networks
by: Plate, Christoph, et al.
Published: (2025)

Convergence rates for gradient descent in the training of overparameterized artificial neural networks with piecewise affine activation
by: Jentzen, Arnulf, et al.
Published: (2021)

Averaged Adam accelerates stochastic optimization in the training of deep neural network approximations for partial differential equation and optimal control problems
by: Dereich, Steffen, et al.
Published: (2025)

Path-conditioned training: a principled way to rescale ReLU neural networks
by: Lebeurrier, Arthur, et al.
Published: (2026)

Local Lipschitz Constant Computation of ReLU-FNNs: Upper Bound Computation with Exactness Verification
by: Ebihara, Yoshio, et al.
Published: (2023)

Non-convergence to global minimizers in data driven supervised deep learning: Adam and stochastic gradient descent optimization provably fail to converge to global minimizers in the training of deep neural networks with ReLU activation
by: Do, Thang, et al.
Published: (2024)

On Bounds for Norms of Reparameterized ReLU Artificial Neural Network Parameters: Sums of Fractional Powers of the Lipschitz Norm Control the Network Parameter Vector
by: Arnulf Jentzen, et al.
Published: (2025)

Convergence rates for the Adam optimizer
by: Dereich, Steffen, et al.
Published: (2024)

Hidden Minima in Two-Layer ReLU Networks
by: Arjevani, Yossi
Published: (2023)

Architecture independent generalization bounds for overparametrized deep ReLU networks
by: Bapu, Anandatheertha, et al.
Published: (2025)

Why Smooth Stability Assumptions Fail for ReLU Learning
by: Katende, Ronald
Published: (2025)

Convex Formulations for Training Two-Layer ReLU Neural Networks
by: Prakhya, Karthik, et al.
Published: (2024)

An Efficient Alternating Algorithm for ReLU-based Symmetric Matrix Decomposition
by: Wang, Qingsong
Published: (2025)

Computational Tradeoffs of Optimization-Based Bound Tightening in ReLU Networks
by: Badilla, Fabian, et al.
Published: (2023)

A Complete Set of Quadratic Constraints for Repeated ReLU and Generalizations
by: Noori, Sahel Vahedi, et al.
Published: (2024)

Provable Accelerated Convergence of Nesterov's Momentum for Deep ReLU Neural Networks
by: Liao, Fangshuo, et al.
Published: (2023)

ODE approximation for the Adam algorithm: General and overparametrized setting
by: Dereich, Steffen, et al.
Published: (2025)

ReLU Networks for Model Predictive Control: Network Complexity and Performance Guarantees
by: Li, Xingchen, et al.
Published: (2026)

Stability and Performance Analysis of Discrete-Time ReLU Recurrent Neural Networks
by: Noori, Sahel Vahedi, et al.
Published: (2024)

Interpretable global minima of deep ReLU neural networks on sequentially separable data
by: Chen, Thomas, et al.
Published: (2024)

Deep neural networks with ReLU, leaky ReLU, and softplus activation provably overcome the curse of dimensionality for Kolmogorov partial differential equations with Lipschitz nonlinearities in the $L^p$-sense
by: Ackermann, Julia, et al.
Published: (2023)

Non-asymptotic convergence analysis of the stochastic gradient Hamiltonian Monte Carlo algorithm with discontinuous stochastic gradient with applications to training of ReLU neural networks
by: Liang, Luxu, et al.
Published: (2024)

Convex Relaxations of ReLU Neural Networks Approximate Global Optima in Polynomial Time
by: Kim, Sungyoon, et al.
Published: (2024)

Approximation with Random Shallow ReLU Networks with Applications to Model Reference Adaptive Control
by: Lamperski, Andrew, et al.
Published: (2024)

Pruning for efficient deterministic global optimization over trained ReLU neural networks
by: Lastrucci, Giacomo, et al.
Published: (2026)

Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses
by: Dereich, Steffen, et al.
Published: (2024)

PADAM: Parallel averaged Adam reduces the error for stochastic optimization in scientific machine learning
by: Jentzen, Arnulf, et al.
Published: (2025)

Adversarial Training of Two-Layer Polynomial and ReLU Activation Networks via Convex Optimization
by: Kuelbs, Daniel, et al.
Published: (2024)

Homotopy Relaxation Training Algorithms for Infinite-Width Two-Layer ReLU Neural Networks
by: Yang, Yahong, et al.
Published: (2023)

Neural Collapse under Gradient Flow on Shallow ReLU Networks for Orthogonally Separable Data
by: Min, Hancheng, et al.
Published: (2025)

On the existence of optimal shallow feedforward networks with ReLU activation
by: Dereich, Steffen, et al.
Published: (2023)

Deep neural networks with ReLU, leaky ReLU, and softplus activation provably overcome the curse of dimensionality for space-time solutions of semilinear partial differential equations
by: Ackermann, Julia, et al.
Published: (2024)

Function Gradient Approximation with Random Shallow ReLU Networks with Control Applications
by: Lamperski, Andrew, et al.
Published: (2024)

How Does the ReLU Activation Affect the Implicit Bias of Gradient Descent on High-dimensional Neural Network Regression?
by: Lai, Kuo-Wei, et al.
Published: (2026)

Nonlinear Dynamics In Optimization Landscape of Shallow Neural Networks with Tunable Leaky ReLU
by: Liu, Jingzhou
Published: (2025)

On the growth of the parameters of approximating ReLU neural networks
by: Morina, Erion, et al.
Published: (2024)