Copertina

Depth Degeneracy in Neural Networks: Vanishing Angles in Fully Connected ReLU Networks on Initialization

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Jakub, Cameron, Nica, Mihai
Natura:	Preprint
Pubblicazione:	2023
Soggetti:	Machine Learning Probability
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Optimal Initialization in Depth: Lyapunov Initialization and Limit Theorems for Deep Leaky ReLU Networks
di: Kogler, Constantin, et al.
Pubblicazione: (2026)

Network Degeneracy as an Indicator of Training Performance: Comparing Finite and Infinite Width Angle Predictions
di: Jakub, Cameron, et al.
Pubblicazione: (2023)

Random ReLU Neural Networks as Non-Gaussian Processes
di: Parhi, Rahul, et al.
Pubblicazione: (2024)

Large Deviations of Gaussian Neural Networks with ReLU activation
di: Vogel, Quirin
Pubblicazione: (2024)

Stably unactivated neurons in ReLU neural networks
di: Brownlowe, Natalie, et al.
Pubblicazione: (2024)

On the Expressiveness of Rational ReLU Neural Networks With Bounded Depth
di: Averkov, Gennadiy, et al.
Pubblicazione: (2025)

On the Depth of Monotone ReLU Neural Networks and ICNNs
di: Bakaev, Egor, et al.
Pubblicazione: (2025)

Injectivity of ReLU networks: perspectives from statistical physics
di: Maillard, Antoine, et al.
Pubblicazione: (2023)

Optimized Weight Initialization on the Stiefel Manifold for Deep ReLU Neural Networks
di: Lee, Hyungu, et al.
Pubblicazione: (2025)

Multilevel Picard approximations and deep neural networks with ReLU, leaky ReLU, and softplus activation overcome the curse of dimensionality when approximating semilinear parabolic partial differential equations in $L^p$-sense
di: Neufeld, Ariel, et al.
Pubblicazione: (2024)

Uncertainty Quantification with Bayesian Higher Order ReLU KANs
di: Giroux, James, et al.
Pubblicazione: (2024)

Towards Lower Bounds on the Depth of ReLU Neural Networks
di: Hertrich, Christoph, et al.
Pubblicazione: (2021)

Pathwise Explanation of ReLU Neural Networks
di: Lim, Seongwoo, et al.
Pubblicazione: (2025)

Discrete Functional Geometry of ReLU Networks via ReLU Transition Graphs
di: Dhayalkar, Sahil Rajesh
Pubblicazione: (2025)

Convexity in ReLU Neural Networks: beyond ICNNs?
di: Gagneux, Anne, et al.
Pubblicazione: (2025)

Stochastic Bandits with ReLU Neural Networks
di: Xu, Kan, et al.
Pubblicazione: (2024)

On Space Folds of ReLU Neural Networks
di: Lewandowski, Michal, et al.
Pubblicazione: (2025)

The Geometry of ReLU Networks through the ReLU Transition Graph
di: Dhayalkar, Sahil Rajesh
Pubblicazione: (2025)

Topological Expressivity of ReLU Neural Networks
di: Ergen, Ekin, et al.
Pubblicazione: (2023)

Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization
di: Min, Hancheng, et al.
Pubblicazione: (2023)

Hamiltonian Monte Carlo on ReLU Neural Networks is Inefficient
di: Dinh, Vu C., et al.
Pubblicazione: (2024)

Complexity of Injectivity and Verification of ReLU Neural Networks
di: Froese, Vincent, et al.
Pubblicazione: (2024)

Initialization Matters: On the Benign Overfitting of Two-Layer ReLU CNN with Fully Trainable Layers
di: Shang, Shuning, et al.
Pubblicazione: (2024)

Degeneracy is OK: Logarithmic Regret for Network Revenue Management with Indiscrete Distributions
di: Jiang, Jiashuo, et al.
Pubblicazione: (2022)

Dense ReLU Neural Networks for Temporal-spatial Model
di: Padilla, Carlos Misael Madrid, et al.
Pubblicazione: (2024)

Neural Characteristic Activation Analysis and Geometric Parameterization for ReLU Networks
di: Chen, Wenlin, et al.
Pubblicazione: (2023)

The Effects of Multi-Task Learning on ReLU Neural Network Functions
di: Nakhleh, Julia, et al.
Pubblicazione: (2024)

From Tempered to Benign Overfitting in ReLU Neural Networks
di: Kornowski, Guy, et al.
Pubblicazione: (2023)

Optimal Sets and Solution Paths of ReLU Networks
di: Mishkin, Aaron, et al.
Pubblicazione: (2023)

On Size-Independent Sample Complexity of ReLU Networks
di: Sellke, Mark
Pubblicazione: (2023)

Online Realizable Regression and Applications for ReLU Networks
di: Doron-Arad, Ilan, et al.
Pubblicazione: (2026)

Convex Formulations for Training Two-Layer ReLU Neural Networks
di: Prakhya, Karthik, et al.
Pubblicazione: (2024)

Some Super-approximation Rates of ReLU Neural Networks for Korobov Functions
di: Li, Yuwen, et al.
Pubblicazione: (2025)

ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing
di: Wang, Ziteng, et al.
Pubblicazione: (2024)

A Depth Hierarchy for Computing the Maximum in ReLU Networks via Extremal Graph Theory
di: Safran, Itay
Pubblicazione: (2026)

SurvReLU: Inherently Interpretable Survival Analysis via Deep ReLU Networks
di: Sun, Xiaotong, et al.
Pubblicazione: (2024)

Three Quantization Regimes for ReLU Networks
di: Ou, Weigutian, et al.
Pubblicazione: (2024)

N-ReLU: Zero-Mean Stochastic Extension of ReLU
di: Manik, Md Motaleb Hossen, et al.
Pubblicazione: (2025)

When Are Bias-Free ReLU Networks Effectively Linear Networks?
di: Zhang, Yedi, et al.
Pubblicazione: (2024)

Differential Equation Scaling Limits of Shaped and Unshaped Neural Networks
di: Li, Mufan Bill, et al.
Pubblicazione: (2023)