Cover Image

Optimal Initialization in Depth: Lyapunov Initialization and Limit Theorems for Deep Leaky ReLU Networks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kogler, Constantin, Schwarz, Tassilo, Kittle, Samuel
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Dynamical Systems Probability
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Dimension of contracting on average self-similar measures
by: Kittle, Samuel, et al.
Published: (2025)

Entropy Theory for Random Walks on Lie Groups
by: Kittle, Samuel, et al.
Published: (2025)

On absolute continuity of inhomogeneous and contracting on average self-similar measures
by: Kittle, Samuel, et al.
Published: (2024)

Polynomial Tail Decay for Stationary Measures
by: Kittle, Samuel, et al.
Published: (2025)

Depth Degeneracy in Neural Networks: Vanishing Angles in Fully Connected ReLU Networks on Initialization
by: Jakub, Cameron, et al.
Published: (2023)

Nonlinear Dynamics In Optimization Landscape of Shallow Neural Networks with Tunable Leaky ReLU
by: Liu, Jingzhou
Published: (2025)

Detecting Invariant Manifolds in ReLU-Based RNNs
by: Eisenmann, Lukas, et al.
Published: (2025)

Effective density of non-degenerate random walks on homogeneous spaces
by: Kim, Wooyeon, et al.
Published: (2023)

Injectivity of ReLU networks: perspectives from statistical physics
by: Maillard, Antoine, et al.
Published: (2023)

Random ReLU Neural Networks as Non-Gaussian Processes
by: Parhi, Rahul, et al.
Published: (2024)

Permutation-Invariant Spectral Learning via Dyson Diffusion
by: Schwarz, Tassilo, et al.
Published: (2025)

Stably unactivated neurons in ReLU neural networks
by: Brownlowe, Natalie, et al.
Published: (2024)

Optimized Weight Initialization on the Stiefel Manifold for Deep ReLU Neural Networks
by: Lee, Hyungu, et al.
Published: (2025)

Large Deviations of Gaussian Neural Networks with ReLU activation
by: Vogel, Quirin
Published: (2024)

Geometric regularization of autoencoders via observed stochastic dynamics
by: Hill, Sean, et al.
Published: (2026)

The Minary Primitive of Computational Autopoiesis
by: Connor, Daniel, et al.
Published: (2026)

Random Quadratic Form on a Sphere: Synchronization by Common Noise
by: Engel, Maximilian, et al.
Published: (2026)

A Score Filter Enhanced Data Assimilation Framework for Data-Driven Dynamical Systems
by: Tang, Jingqiao, et al.
Published: (2026)

Bridging discrete and continuous state spaces: Exploring the Ehrenfest process in time-continuous diffusion models
by: Winkler, Ludwig, et al.
Published: (2024)

Residual connections provably mitigate oversmoothing in graph neural networks
by: Chen, Ziang, et al.
Published: (2025)

Characterizing Dynamical Stability of Stochastic Gradient Descent in Overparameterized Learning
by: Chemnitz, Dennis, et al.
Published: (2024)

Divergence-kernel method for linear responses of densities and generative models
by: Ni, Angxiu
Published: (2025)

When and how can inexact generative models still sample from the data manifold?
by: Chandramoorthy, Nisha, et al.
Published: (2025)

Regularity of Lyapunov exponents at one-point Lyapunov spectra: the semisimple case
by: Liu, Yingjian, et al.
Published: (2026)

Integral representation of Lyapunov exponents
by: Barrientos, Pablo G., et al.
Published: (2026)

On the Quenched Functional Central Limit Theorem for Stationary Random Fields under Projective Criteria
by: Reding, Lucas, et al.
Published: (2021)

Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization
by: Min, Hancheng, et al.
Published: (2023)

Complex-Weighted Convolutional Networks: Provable Expressiveness via Complex Diffusion
by: Amado, Cristina López, et al.
Published: (2025)

Central Limit Theorems in Multiplicative Diophantine Approximation
by: Björklund, Michael, et al.
Published: (2026)

Multilevel Picard approximations and deep neural networks with ReLU, leaky ReLU, and softplus activation overcome the curse of dimensionality when approximating semilinear parabolic partial differential equations in $L^p$-sense
by: Neufeld, Ariel, et al.
Published: (2024)

Uncertainty Quantification with Bayesian Higher Order ReLU KANs
by: Giroux, James, et al.
Published: (2024)

The Kinetic Limit of Balanced Neural Networks
by: MacLaurin, James, et al.
Published: (2025)

Data-driven Initial Gap Identification of Piecewise-linear Systems using Sparse Regression and Universal Approximation Theorem
by: Kanki, Ryosuke, et al.
Published: (2026)

Analytic Dependence of the Lyapunov Moment Function and the Projective Stationary Measure for Random Matrix Products
by: Chalhoub, Christopher, et al.
Published: (2025)

Continuous dependence for McKean-Vlasov SDEs under distribution-dependent Lyapunov conditions
by: Ma, Jun, et al.
Published: (2024)

How to Tame Your LLM: Semantic Collapse in Continuous Systems
by: Wyss, C. M.
Published: (2025)

The Mean-Field Dynamics of Transformers
by: Rigollet, Philippe
Published: (2025)

Data-driven approximation of transfer operators for mean-field stochastic differential equations
by: Ioannou, Eirini, et al.
Published: (2025)

A Dynamics-Informed Gaussian Process Framework for 2D Stochastic Navier-Stokes via Quasi-Gaussianity
by: Hamzi, Boumediene, et al.
Published: (2025)

Learning Chaotic Systems and Long-Term Predictions with Neural Jump ODEs
by: Krach, Florian, et al.
Published: (2024)