Image de couverture de livre

Mildly Overparameterized ReLU Networks on Orthogonal Data: Incremental Learning and Implicit Bias

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Town, James, Boursier, Etienne, Lewis, Ben, Englert, Matthias, Lazic, Ranko
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Benignity of loss landscape with weight decay requires both large overparametrization and initialization
par: Boursier, Etienne, et autres
Publié: (2025)

Simplicity bias and optimization threshold in two-layer ReLU networks
par: Boursier, Etienne, et autres
Publié: (2024)

Mildly Overparameterized ReLU Networks Have a Favorable Loss Landscape
par: Karhadkar, Kedar, et autres
Publié: (2023)

Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs
par: Boursier, Etienne, et autres
Publié: (2022)

Asymptotic Smoothing of the Lipschitz Loss Landscape in Overparameterized One-Hidden-Layer ReLU Networks
par: Baturin, Saveliy
Publié: (2026)

Implicit Hypersurface Approximation Capacity in Deep ReLU Networks
par: Vallin, Jonatan, et autres
Publié: (2024)

Implicit Regularization Towards Rank Minimization in ReLU Networks
par: Timor, Nadav, et autres
Publié: (2022)

When Are Bias-Free ReLU Networks Effectively Linear Networks?
par: Zhang, Yedi, et autres
Publié: (2024)

Discrete Functional Geometry of ReLU Networks via ReLU Transition Graphs
par: Dhayalkar, Sahil Rajesh
Publié: (2025)

The Geometry of ReLU Networks through the ReLU Transition Graph
par: Dhayalkar, Sahil Rajesh
Publié: (2025)

Neural Collapse under Gradient Flow on Shallow ReLU Networks for Orthogonally Separable Data
par: Min, Hancheng, et autres
Publié: (2025)

N-ReLU: Zero-Mean Stochastic Extension of ReLU
par: Manik, Md Motaleb Hossen, et autres
Publié: (2025)

The Resurrection of the ReLU
par: Horuz, Coşku Can, et autres
Publié: (2025)

How Does the ReLU Activation Affect the Implicit Bias of Gradient Descent on High-dimensional Neural Network Regression?
par: Lai, Kuo-Wei, et autres
Publié: (2026)

Noisy Interpolation Learning with Shallow Univariate ReLU Networks
par: Joshi, Nirmit, et autres
Publié: (2023)

Saddle-To-Saddle Dynamics in Deep ReLU Networks: Low-Rank Bias in the First Saddle Escape
par: Bantzis, Ioannis, et autres
Publié: (2025)

Pathwise Explanation of ReLU Neural Networks
par: Lim, Seongwoo, et autres
Publié: (2025)

Online Realizable Regression and Applications for ReLU Networks
par: Doron-Arad, Ilan, et autres
Publié: (2026)

Optimal Sets and Solution Paths of ReLU Networks
par: Mishkin, Aaron, et autres
Publié: (2023)

On Size-Independent Sample Complexity of ReLU Networks
par: Sellke, Mark
Publié: (2023)

Convexity in ReLU Neural Networks: beyond ICNNs?
par: Gagneux, Anne, et autres
Publié: (2025)

SurvReLU: Inherently Interpretable Survival Analysis via Deep ReLU Networks
par: Sun, Xiaotong, et autres
Publié: (2024)

The Effects of Multi-Task Learning on ReLU Neural Network Functions
par: Nakhleh, Julia, et autres
Publié: (2024)

Stochastic Bandits with ReLU Neural Networks
par: Xu, Kan, et autres
Publié: (2024)

On Space Folds of ReLU Neural Networks
par: Lewandowski, Michal, et autres
Publié: (2025)

Convergence of Shallow ReLU Networks on Weakly Interacting Data
par: Dana, Léo, et autres
Publié: (2025)

ReLU-KAN: New Kolmogorov-Arnold Networks that Only Need Matrix Addition, Dot Multiplication, and ReLU
par: Qiu, Qi, et autres
Publié: (2024)

Topological Expressivity of ReLU Neural Networks
par: Ergen, Ekin, et autres
Publié: (2023)

Three Quantization Regimes for ReLU Networks
par: Ou, Weigutian, et autres
Publié: (2024)

Geometry of Singular Foliations and Learning Manifolds in ReLU Networks via the Data Information Matrix
par: Tron, Eliot, et autres
Publié: (2024)

Hidden Minima in Two-Layer ReLU Networks
par: Arjevani, Yossi
Publié: (2023)

On the Local Complexity of Linear Regions in Deep ReLU Networks
par: Patel, Niket, et autres
Publié: (2024)

ReLU Networks as Random Functions: Their Distribution in Probability Space
par: Chaudhari, Shreyas, et autres
Publié: (2025)

Training a Two Layer ReLU Network Analytically
par: Barbu, Adrian
Publié: (2023)

Hamiltonian Monte Carlo on ReLU Neural Networks is Inefficient
par: Dinh, Vu C., et autres
Publié: (2024)

The Symmetries of Three-Layer ReLU Networks
par: Gegenfurtner, Johanna Marie, et autres
Publié: (2026)

Brownian ReLU(Br-ReLU): A New Activation Function for a Long-Short Term Memory (LSTM) Network
par: Awiakye-Marfo, George, et autres
Publié: (2026)

Complexity of One-Dimensional ReLU DNNs
par: Kogan, Jonathan, et autres
Publié: (2025)

Symmetric Matrix Completion with ReLU Sampling
par: Liu, Huikang, et autres
Publié: (2024)

Random ReLU Neural Networks as Non-Gaussian Processes
par: Parhi, Rahul, et autres
Publié: (2024)