Cover Image

Zorro: A Flexible and Differentiable Parametric Family of Activation Functions That Extends ReLU and GELU

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Roodschild, Matias, Gotay-Sardiñas, Jorge, Jimenez, Victor A., Will, Adrian
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Neural and Evolutionary Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The Geometry of ReLU Networks through the ReLU Transition Graph
by: Dhayalkar, Sahil Rajesh
Published: (2025)

On Space Folds of ReLU Neural Networks
by: Lewandowski, Michal, et al.
Published: (2025)

ReLU-KAN: New Kolmogorov-Arnold Networks that Only Need Matrix Addition, Dot Multiplication, and ReLU
by: Qiu, Qi, et al.
Published: (2024)

From Tempered to Benign Overfitting in ReLU Neural Networks
by: Kornowski, Guy, et al.
Published: (2023)

Causal Explanations from the Geometric Properties of ReLU Neural Networks
by: Woods, Hector, et al.
Published: (2026)

Expressivity and Approximation Properties of Deep Neural Networks with ReLU$^k$ Activation
by: He, Juncai, et al.
Published: (2023)

Sign Gradient Descent-based Neuronal Dynamics: ANN-to-SNN Conversion Beyond ReLU Network
by: Oh, Hyunseok, et al.
Published: (2024)

On the Depth of Monotone ReLU Neural Networks and ICNNs
by: Bakaev, Egor, et al.
Published: (2025)

Towards Lower Bounds on the Depth of ReLU Neural Networks
by: Hertrich, Christoph, et al.
Published: (2021)

Provable Unlearning with Gradient Ascent on Two-Layer ReLU Neural Networks
by: Melamed, Odelia, et al.
Published: (2025)

A Significantly Better Class of Activation Functions Than ReLU Like Activation Functions
by: Noel, Mathew Mithra, et al.
Published: (2024)

Deep Learning using Rectified Linear Units (ReLU)
by: Agarap, Abien Fred
Published: (2018)

Approximation Rates and VC-Dimension Bounds for (P)ReLU MLP Mixture of Experts
by: Kratsios, Anastasis, et al.
Published: (2024)

The Computational Complexity of Counting Linear Regions in ReLU Neural Networks
by: Stargalla, Moritz, et al.
Published: (2025)

Geometry-induced Regularization in Deep ReLU Neural Networks
by: Bona-Pellissier, Joachim, et al.
Published: (2024)

ReLU Neural Networks of Polynomial Size for Exact Maximum Flow Computation
by: Hertrich, Christoph, et al.
Published: (2021)

Competition-based Adaptive ReLU for Deep Neural Networks
by: Chen, Junjia, et al.
Published: (2024)

Robustness of Deep ReLU Networks to Misclassification of High-Dimensional Data
by: Kůrková, Věra
Published: (2026)

Efficient Approximation to Analytic and $L^p$ functions by Height-Augmented ReLU Networks
by: Li, ZeYu, et al.
Published: (2026)

Learnable Activation Functions in Physics-Informed Neural Networks for Solving Partial Differential Equations
by: Farea, Afrah, et al.
Published: (2024)

APTx: better activation function than MISH, SWISH, and ReLU's variants used in deep learning
by: Kumar, Ravin
Published: (2022)

Deriving Activation Functions Using Integration
by: Huang, Allen Hao, et al.
Published: (2024)

Topology-Aware Activation Functions in Neural Networks
by: Snopov, Pavel, et al.
Published: (2025)

Expanded Gating Ranges Improve Activation Functions
by: Huang, Allen Hao
Published: (2024)

Flexible inference for animal learning rules using neural networks
by: Liu, Yuhan Helena, et al.
Published: (2025)

Stable and Robust Deep Learning By Hyperbolic Tangent Exponential Linear Unit (TeLU)
by: Fernandez, Alfredo, et al.
Published: (2024)

Activation Functions for "A Feedforward Unitary Equivariant Neural Network"
by: Ma, Pui-Wai
Published: (2024)

Adaptive Activation Functions for Predictive Modeling with Sparse Experimental Data
by: Pourkamali-Anaraki, Farhad, et al.
Published: (2024)

Parametric-Task MAP-Elites
by: Anne, Timothée, et al.
Published: (2024)

APALU: A Trainable, Adaptive Activation Function for Deep Learning Networks
by: Subramanian, Barathi, et al.
Published: (2024)

A More Accurate Approximation of Activation Function with Few Spikes Neurons
by: Jeong, Dayena, et al.
Published: (2024)

Evolving Multi-Channel Confidence-Aware Activation Functions for Missing Data with Channel Propagation
by: Sani, Naeem Shahabi, et al.
Published: (2026)

Sharp Lower Bounds on Interpolation by Deep ReLU Neural Networks at Irregularly Spaced Data
by: Siegel, Jonathan W.
Published: (2023)

MOELIGA: a multi-objective evolutionary approach for feature selection with local improvement
by: Vignolo, Leandro, et al.
Published: (2026)

Higher-order-ReLU-KANs (HRKANs) for solving physics-informed neural networks (PINNs) more accurately, robustly and faster
by: So, Chi Chiu, et al.
Published: (2024)

On the Principles of ReLU Networks with One Hidden Layer
by: Huang, Changcun
Published: (2024)

Hysteresis Activation Function for Efficient Inference
by: Kimhi, Moshe, et al.
Published: (2024)

($\boldsymbolθ_l, \boldsymbolθ_u$)-Parametric Multi-Task Optimization: Joint Search in Solution and Infinite Task Spaces
by: Wei, Tingyang, et al.
Published: (2025)

PReLU: Yet Another Single-Layer Solution to the XOR Problem
by: Pinto, Rafael C., et al.
Published: (2024)

Flexible and Efficient Surrogate Gradient Modeling with Forward Gradient Injection
by: Otte, Sebastian
Published: (2024)