Cover Image

Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Min, Hancheng, Mallada, Enrique, Vidal, René
Format:	Preprint
Published:	2023
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Neural Collapse under Gradient Flow on Shallow ReLU Networks for Orthogonally Separable Data
by: Min, Hancheng, et al.
Published: (2025)

A Local Polyak-Lojasiewicz and Descent Lemma of Gradient Descent For Overparametrized Linear Models
by: Xu, Ziqing, et al.
Published: (2025)

Discrete Functional Geometry of ReLU Networks via ReLU Transition Graphs
by: Dhayalkar, Sahil Rajesh
Published: (2025)

The Geometry of ReLU Networks through the ReLU Transition Graph
by: Dhayalkar, Sahil Rajesh
Published: (2025)

Two-hidden-layer ReLU neural networks and finite elements
by: Jin, Pengzhan
Published: (2024)

Hidden Minima in Two-Layer ReLU Networks
by: Arjevani, Yossi
Published: (2023)

Training a Two Layer ReLU Network Analytically
by: Barbu, Adrian
Published: (2023)

Optimized Weight Initialization on the Stiefel Manifold for Deep ReLU Neural Networks
by: Lee, Hyungu, et al.
Published: (2025)

Memorization Capacity for Additive Fine-Tuning with Small ReLU Networks
by: Sohn, Jy-yong, et al.
Published: (2024)

N-ReLU: Zero-Mean Stochastic Extension of ReLU
by: Manik, Md Motaleb Hossen, et al.
Published: (2025)

Injectivity of ReLU-layers: Tools from Frame Theory
by: Haider, Daniel, et al.
Published: (2024)

Initialization Matters: On the Benign Overfitting of Two-Layer ReLU CNN with Fully Trainable Layers
by: Shang, Shuning, et al.
Published: (2024)

The Resurrection of the ReLU
by: Horuz, Coşku Can, et al.
Published: (2025)

Optimal Initialization in Depth: Lyapunov Initialization and Limit Theorems for Deep Leaky ReLU Networks
by: Kogler, Constantin, et al.
Published: (2026)

Depth Degeneracy in Neural Networks: Vanishing Angles in Fully Connected ReLU Networks on Initialization
by: Jakub, Cameron, et al.
Published: (2023)

Benign Overfitting for Regression with Trained Two-Layer ReLU Networks
by: Park, Junhyung, et al.
Published: (2024)

Convex Formulations for Training Two-Layer ReLU Neural Networks
by: Prakhya, Karthik, et al.
Published: (2024)

Simplicity bias and optimization threshold in two-layer ReLU networks
by: Boursier, Etienne, et al.
Published: (2024)

Pathwise Explanation of ReLU Neural Networks
by: Lim, Seongwoo, et al.
Published: (2025)

Optimal Sets and Solution Paths of ReLU Networks
by: Mishkin, Aaron, et al.
Published: (2023)

On Size-Independent Sample Complexity of ReLU Networks
by: Sellke, Mark
Published: (2023)

Online Realizable Regression and Applications for ReLU Networks
by: Doron-Arad, Ilan, et al.
Published: (2026)

Convexity in ReLU Neural Networks: beyond ICNNs?
by: Gagneux, Anne, et al.
Published: (2025)

SurvReLU: Inherently Interpretable Survival Analysis via Deep ReLU Networks
by: Sun, Xiaotong, et al.
Published: (2024)

Stochastic Bandits with ReLU Neural Networks
by: Xu, Kan, et al.
Published: (2024)

On Space Folds of ReLU Neural Networks
by: Lewandowski, Michal, et al.
Published: (2025)

Can Implicit Bias Imply Adversarial Robustness?
by: Min, Hancheng, et al.
Published: (2024)

Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networks
by: Collins, Liam, et al.
Published: (2023)

Compelling ReLU Networks to Exhibit Exponentially Many Linear Regions at Initialization and During Training
by: Milkert, Max, et al.
Published: (2023)

ReLU-KAN: New Kolmogorov-Arnold Networks that Only Need Matrix Addition, Dot Multiplication, and ReLU
by: Qiu, Qi, et al.
Published: (2024)

Explicit integral representations and quantitative bounds for two-layer ReLU networks
by: Lee, Anthony
Published: (2026)

Topological Expressivity of ReLU Neural Networks
by: Ergen, Ekin, et al.
Published: (2023)

Three Quantization Regimes for ReLU Networks
by: Ou, Weigutian, et al.
Published: (2024)

Two-Dimensional Deep ReLU CNN Approximation for Korobov Functions: A Constructive Approach
by: Fang, Qin, et al.
Published: (2025)

Noisy Interpolation Learning with Shallow Univariate ReLU Networks
by: Joshi, Nirmit, et al.
Published: (2023)

On the Local Complexity of Linear Regions in Deep ReLU Networks
by: Patel, Niket, et al.
Published: (2024)

ReLU Networks as Random Functions: Their Distribution in Probability Space
by: Chaudhari, Shreyas, et al.
Published: (2025)

Implicit Hypersurface Approximation Capacity in Deep ReLU Networks
by: Vallin, Jonatan, et al.
Published: (2024)

Hamiltonian Monte Carlo on ReLU Neural Networks is Inefficient
by: Dinh, Vu C., et al.
Published: (2024)

Implicit Regularization Towards Rank Minimization in ReLU Networks
by: Timor, Nadav, et al.
Published: (2022)