Cover Image

The effect of Leaky ReLUs on the training and generalization of overparameterized networks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Guo, Yinglong, Li, Shaohan, Lerman, Gilad
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Transformer Approximations from ReLUs
by: Hu, Jerry Yao-Chieh, et al.
Published: (2026)

ReLUs Are Sufficient for Learning Implicit Neural Representations
by: Shenouda, Joseph, et al.
Published: (2024)

From Tempered to Benign Overfitting in ReLU Neural Networks
by: Kornowski, Guy, et al.
Published: (2023)

Leaky ReLUs That Differ in Forward and Backward Pass Facilitate Activation Maximization in Deep Neural Networks
by: Linse, Christoph, et al.
Published: (2024)

Nonlinear Dynamics In Optimization Landscape of Shallow Neural Networks with Tunable Leaky ReLU
by: Liu, Jingzhou
Published: (2025)

Optimal Initialization in Depth: Lyapunov Initialization and Limit Theorems for Deep Leaky ReLU Networks
by: Kogler, Constantin, et al.
Published: (2026)

Minimum Width of Leaky-ReLU Neural Networks for Uniform Universal Approximation
by: Li, Li'ang, et al.
Published: (2023)

Spanning Multi-Asset Payoffs With ReLUs
by: Bossu, Sébastien, et al.
Published: (2024)

Spanning Multi‐Asset Payoffs With ReLUs
by: Sébastien Bossu, et al.
Published: (2025)

Theoretical Guarantees for the Subspace-Constrained Tyler's Estimator
by: Lerman, Gilad, et al.
Published: (2024)

Provable Unlearning with Gradient Ascent on Two-Layer ReLU Neural Networks
by: Melamed, Odelia, et al.
Published: (2025)

Convergence rates for gradient descent in the training of overparameterized artificial neural networks with piecewise affine activation
by: Jentzen, Arnulf, et al.
Published: (2021)

Path-conditioned training: a principled way to rescale ReLU neural networks
by: Lebeurrier, Arthur, et al.
Published: (2026)

Understanding the role of depth in the neural tangent kernel for overparameterized neural networks
by: St-Arnaud, William, et al.
Published: (2025)

Constraining the outputs of ReLU neural networks
by: Alexandr, Yulia, et al.
Published: (2025)

Toric geometry of ReLU neural networks
by: Fu, Yaoying
Published: (2025)

Stably unactivated neurons in ReLU neural networks
by: Brownlowe, Natalie, et al.
Published: (2024)

Discrete Functional Geometry of ReLU Networks via ReLU Transition Graphs
by: Dhayalkar, Sahil Rajesh
Published: (2025)

Stein Discrepancy for Unsupervised Domain Adaptation
by: von Seeger, Anneke, et al.
Published: (2025)

Benign overfitting in leaky ReLU networks with moderate input dimension
by: Karhadkar, Kedar, et al.
Published: (2024)

Generalization analysis with deep ReLU networks for metric and similarity learning
by: Zhou, Junyu, et al.
Published: (2024)

Simplicity bias and optimization threshold in two-layer ReLU networks
by: Boursier, Etienne, et al.
Published: (2024)

N-ReLU: Zero-Mean Stochastic Extension of ReLU
by: Manik, Md Motaleb Hossen, et al.
Published: (2025)

Weighted variation spaces and approximation by shallow ReLU networks
by: DeVore, Ronald, et al.
Published: (2023)

An analysis of optimization problems involving ReLU neural networks
by: Plate, Christoph, et al.
Published: (2025)

Topological obstruction to the training of shallow ReLU neural networks
by: Nurisso, Marco, et al.
Published: (2024)

The Resurrection of the ReLU
by: Horuz, Coşku Can, et al.
Published: (2025)

Optimal Rates for Generalization of Gradient Descent for Deep ReLU Classification
by: Li, Yuanfan, et al.
Published: (2025)

Explicit integral representations and quantitative bounds for two-layer ReLU networks
by: Lee, Anthony
Published: (2026)

Minimum width for universal approximation using ReLU networks on compact domain
by: Kim, Namjun, et al.
Published: (2023)

Artificial Intelligence and Algorithmic Price Collusion in Two-sided Markets
by: Chica, Cristian, et al.
Published: (2024)

Learning to Charge More: A Theoretical Study of Collusion by Q-Learning Agents
by: Chica, Cristian, et al.
Published: (2025)

Competition and Collusion in Two-Sided Markets with an Outside Option
by: Chica, Cristian, et al.
Published: (2025)

Global Convergence of Iteratively Reweighted Least Squares for Robust Subspace Recovery
by: Lerman, Gilad, et al.
Published: (2025)

Architecture independent generalization bounds for overparametrized deep ReLU networks
by: Bapu, Anandatheertha, et al.
Published: (2025)

Two-hidden-layer ReLU neural networks and finite elements
by: Jin, Pengzhan
Published: (2024)

Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs
by: Boursier, Etienne, et al.
Published: (2022)

The Geometry of ReLU Networks through the ReLU Transition Graph
by: Dhayalkar, Sahil Rajesh
Published: (2025)

On the algorithmic construction of deep ReLU networks
by: Huybrechs, Daan
Published: (2025)

Injectivity of ReLU networks: perspectives from statistical physics
by: Maillard, Antoine, et al.
Published: (2023)