Cover Image

Better NTK Conditioning: A Free Lunch from (ReLU) Nonlinear Activation in Wide Neural Networks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Chaoyue, Bi, Han, Hui, Like, Liu, Xiao
Format:	Preprint
Published:	2023
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Beyond ReLU: How Activations Affect Neural Kernels and Random Wide Networks
by: Holzmüller, David, et al.
Published: (2025)

Label-NTK Alignments and A Tighter Convergence Bound in the NTK Regime
by: Marreddy, Ruchirinkil, et al.
Published: (2026)

Neural Characteristic Activation Analysis and Geometric Parameterization for ReLU Networks
by: Chen, Wenlin, et al.
Published: (2023)

Nonlinear Dynamics In Optimization Landscape of Shallow Neural Networks with Tunable Leaky ReLU
by: Liu, Jingzhou
Published: (2025)

Topological Signatures of ReLU Neural Network Activation Patterns
by: Bosca, Vicente, et al.
Published: (2025)

Discrete Functional Geometry of ReLU Networks via ReLU Transition Graphs
by: Dhayalkar, Sahil Rajesh
Published: (2025)

Pathwise Explanation of ReLU Neural Networks
by: Lim, Seongwoo, et al.
Published: (2025)

Convexity in ReLU Neural Networks: beyond ICNNs?
by: Gagneux, Anne, et al.
Published: (2025)

Stochastic Bandits with ReLU Neural Networks
by: Xu, Kan, et al.
Published: (2024)

On Space Folds of ReLU Neural Networks
by: Lewandowski, Michal, et al.
Published: (2025)

The Geometry of ReLU Networks through the ReLU Transition Graph
by: Dhayalkar, Sahil Rajesh
Published: (2025)

Topological Expressivity of ReLU Neural Networks
by: Ergen, Ekin, et al.
Published: (2023)

Brownian ReLU(Br-ReLU): A New Activation Function for a Long-Short Term Memory (LSTM) Network
by: Awiakye-Marfo, George, et al.
Published: (2026)

Activation-Descent Regularization for Input Optimization of ReLU Networks
by: Yu, Hongzhan, et al.
Published: (2024)

Deep Network Approximation: Beyond ReLU to Diverse Activation Functions
by: Zhang, Shijun, et al.
Published: (2023)

When Are Bias-Free ReLU Networks Effectively Linear Networks?
by: Zhang, Yedi, et al.
Published: (2024)

ReCA: A Parametric ReLU Composite Activation Function
by: Chidiac, John, et al.
Published: (2025)

ReLU$^2$ Wins: Discovering Efficient Activation Functions for Sparse LLMs
by: Zhang, Zhengyan, et al.
Published: (2024)

Hamiltonian Monte Carlo on ReLU Neural Networks is Inefficient
by: Dinh, Vu C., et al.
Published: (2024)

N-ReLU: Zero-Mean Stochastic Extension of ReLU
by: Manik, Md Motaleb Hossen, et al.
Published: (2025)

Expressivity and Approximation Properties of Deep Neural Networks with ReLU$^k$ Activation
by: He, Juncai, et al.
Published: (2023)

Random ReLU Neural Networks as Non-Gaussian Processes
by: Parhi, Rahul, et al.
Published: (2024)

The Resurrection of the ReLU
by: Horuz, Coşku Can, et al.
Published: (2025)

Complexity of Injectivity and Verification of ReLU Neural Networks
by: Froese, Vincent, et al.
Published: (2024)

Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Study
by: Liu, Hao, et al.
Published: (2024)

Dense ReLU Neural Networks for Temporal-spatial Model
by: Padilla, Carlos Misael Madrid, et al.
Published: (2024)

On the Expressiveness of Rational ReLU Neural Networks With Bounded Depth
by: Averkov, Gennadiy, et al.
Published: (2025)

The Effects of Multi-Task Learning on ReLU Neural Network Functions
by: Nakhleh, Julia, et al.
Published: (2024)

Symmetric Matrix Completion with ReLU Sampling
by: Liu, Huikang, et al.
Published: (2024)

Confidence Interval Construction and Conditional Variance Estimation with Dense ReLU Networks
by: Padilla, Carlos Misael Madrid, et al.
Published: (2024)

A Momentum Accelerated Algorithm for ReLU-based Nonlinear Matrix Decomposition
by: Wang, Qingsong, et al.
Published: (2024)

From Tempered to Benign Overfitting in ReLU Neural Networks
by: Kornowski, Guy, et al.
Published: (2023)

On the Depth of Monotone ReLU Neural Networks and ICNNs
by: Bakaev, Egor, et al.
Published: (2025)

Causal Explanations from the Geometric Properties of ReLU Neural Networks
by: Woods, Hector, et al.
Published: (2026)

A Significantly Better Class of Activation Functions Than ReLU Like Activation Functions
by: Noel, Mathew Mithra, et al.
Published: (2024)

Optimal Sets and Solution Paths of ReLU Networks
by: Mishkin, Aaron, et al.
Published: (2023)

On Size-Independent Sample Complexity of ReLU Networks
by: Sellke, Mark
Published: (2023)

Online Realizable Regression and Applications for ReLU Networks
by: Doron-Arad, Ilan, et al.
Published: (2026)

SurvReLU: Inherently Interpretable Survival Analysis via Deep ReLU Networks
by: Sun, Xiaotong, et al.
Published: (2024)

Optimized Weight Initialization on the Stiefel Manifold for Deep ReLU Neural Networks
by: Lee, Hyungu, et al.
Published: (2025)