Cover Image

Deep Network Approximation: Beyond ReLU to Diverse Activation Functions

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Shijun, Lu, Jianfeng, Zhao, Hongkai
Format:	Preprint
Published:	2023
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Implicit Hypersurface Approximation Capacity in Deep ReLU Networks
by: Vallin, Jonatan, et al.
Published: (2024)

Discrete Functional Geometry of ReLU Networks via ReLU Transition Graphs
by: Dhayalkar, Sahil Rajesh
Published: (2025)

Expressivity and Approximation Properties of Deep Neural Networks with ReLU$^k$ Activation
by: He, Juncai, et al.
Published: (2023)

ReCA: A Parametric ReLU Composite Activation Function
by: Chidiac, John, et al.
Published: (2025)

Sobolev Approximation of Deep ReLU Networks in Log-Barron Space
by: Song, Changhoon, et al.
Published: (2026)

Brownian ReLU(Br-ReLU): A New Activation Function for a Long-Short Term Memory (LSTM) Network
by: Awiakye-Marfo, George, et al.
Published: (2026)

SurvReLU: Inherently Interpretable Survival Analysis via Deep ReLU Networks
by: Sun, Xiaotong, et al.
Published: (2024)

Beyond ReLU: Chebyshev-DQN for Enhanced Deep Q-Networks
by: Yazdannik, Saman, et al.
Published: (2025)

Beyond ReLU: How Activations Affect Neural Kernels and Random Wide Networks
by: Holzmüller, David, et al.
Published: (2025)

The Geometry of ReLU Networks through the ReLU Transition Graph
by: Dhayalkar, Sahil Rajesh
Published: (2025)

Two-Dimensional Deep ReLU CNN Approximation for Korobov Functions: A Constructive Approach
by: Fang, Qin, et al.
Published: (2025)

Activation-Descent Regularization for Input Optimization of ReLU Networks
by: Yu, Hongzhan, et al.
Published: (2024)

Neural Characteristic Activation Analysis and Geometric Parameterization for ReLU Networks
by: Chen, Wenlin, et al.
Published: (2023)

On the Local Complexity of Linear Regions in Deep ReLU Networks
by: Patel, Niket, et al.
Published: (2024)

ReLU Networks as Random Functions: Their Distribution in Probability Space
by: Chaudhari, Shreyas, et al.
Published: (2025)

N-ReLU: Zero-Mean Stochastic Extension of ReLU
by: Manik, Md Motaleb Hossen, et al.
Published: (2025)

On Polynomial Approximations for Privacy-Preserving and Verifiable ReLU Networks
by: Ali, Ramy E., et al.
Published: (2020)

Function Gradient Approximation with Random Shallow ReLU Networks with Control Applications
by: Lamperski, Andrew, et al.
Published: (2024)

ReLU$^2$ Wins: Discovering Efficient Activation Functions for Sparse LLMs
by: Zhang, Zhengyan, et al.
Published: (2024)

Covering Numbers for Deep ReLU Networks with Applications to Function Approximation and Nonparametric Regression
by: Ou, Weigutian, et al.
Published: (2024)

Approximating Simple ReLU Networks based on Spectral Decomposition of Fisher Information
by: Ho, Ka Long Keith, et al.
Published: (2025)

The Resurrection of the ReLU
by: Horuz, Coşku Can, et al.
Published: (2025)

Some Super-approximation Rates of ReLU Neural Networks for Korobov Functions
by: Li, Yuwen, et al.
Published: (2025)

Linearization of ReLU Activation Function for Neural Network-Embedded Optimization: Optimal Day-Ahead Energy Scheduling
by: Zhao, Cunzhi, et al.
Published: (2023)

The Effects of Multi-Task Learning on ReLU Neural Network Functions
by: Nakhleh, Julia, et al.
Published: (2024)

Topological Signatures of ReLU Neural Network Activation Patterns
by: Bosca, Vicente, et al.
Published: (2025)

Higher Order Approximation Rates for ReLU CNNs in Korobov Spaces
by: Li, Yuwen, et al.
Published: (2025)

Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Study
by: Liu, Hao, et al.
Published: (2024)

Pathwise Explanation of ReLU Neural Networks
by: Lim, Seongwoo, et al.
Published: (2025)

Optimal Sets and Solution Paths of ReLU Networks
by: Mishkin, Aaron, et al.
Published: (2023)

On Size-Independent Sample Complexity of ReLU Networks
by: Sellke, Mark
Published: (2023)

Online Realizable Regression and Applications for ReLU Networks
by: Doron-Arad, Ilan, et al.
Published: (2026)

Convexity in ReLU Neural Networks: beyond ICNNs?
by: Gagneux, Anne, et al.
Published: (2025)

Optimized Weight Initialization on the Stiefel Manifold for Deep ReLU Neural Networks
by: Lee, Hyungu, et al.
Published: (2025)

Why ReLU? A Bit-Model Dichotomy for Deep Network Training
by: Doron-Arad, Ilan, et al.
Published: (2026)

When Are Bias-Free ReLU Networks Effectively Linear Networks?
by: Zhang, Yedi, et al.
Published: (2024)

Component-based Sketching for Deep ReLU Nets
by: Wang, Di, et al.
Published: (2024)

Stochastic Bandits with ReLU Neural Networks
by: Xu, Kan, et al.
Published: (2024)

On Space Folds of ReLU Neural Networks
by: Lewandowski, Michal, et al.
Published: (2025)

ReLU-KAN: New Kolmogorov-Arnold Networks that Only Need Matrix Addition, Dot Multiplication, and ReLU
by: Qiu, Qi, et al.
Published: (2024)