Cover Image

SwishReLU: A Unified Approach to Activation Functions for Enhanced Deep Neural Networks Performance

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Rahman, Jamshaid Ul, Zulfiqar, Rubiqa, Khan, Asad, Nimra
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

DiffGrad for Physics-Informed Neural Networks
by: Rahman, Jamshaid Ul, et al.
Published: (2024)

Swish-T : Enhancing Swish Activation with Tanh Bias for Improved Neural Network Performance
by: Seo, Youngmin, et al.
Published: (2024)

Deep Network Approximation: Beyond ReLU to Diverse Activation Functions
by: Zhang, Shijun, et al.
Published: (2023)

Neural Characteristic Activation Analysis and Geometric Parameterization for ReLU Networks
by: Chen, Wenlin, et al.
Published: (2023)

ReCA: A Parametric ReLU Composite Activation Function
by: Chidiac, John, et al.
Published: (2025)

Expressivity and Approximation Properties of Deep Neural Networks with ReLU$^k$ Activation
by: He, Juncai, et al.
Published: (2023)

Discrete Functional Geometry of ReLU Networks via ReLU Transition Graphs
by: Dhayalkar, Sahil Rajesh
Published: (2025)

The Effects of Multi-Task Learning on ReLU Neural Network Functions
by: Nakhleh, Julia, et al.
Published: (2024)

Brownian ReLU(Br-ReLU): A New Activation Function for a Long-Short Term Memory (LSTM) Network
by: Awiakye-Marfo, George, et al.
Published: (2026)

Beyond ReLU: How Activations Affect Neural Kernels and Random Wide Networks
by: Holzmüller, David, et al.
Published: (2025)

SurvReLU: Inherently Interpretable Survival Analysis via Deep ReLU Networks
by: Sun, Xiaotong, et al.
Published: (2024)

Topological Signatures of ReLU Neural Network Activation Patterns
by: Bosca, Vicente, et al.
Published: (2025)

Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Study
by: Liu, Hao, et al.
Published: (2024)

Optimized Weight Initialization on the Stiefel Manifold for Deep ReLU Neural Networks
by: Lee, Hyungu, et al.
Published: (2025)

Some Super-approximation Rates of ReLU Neural Networks for Korobov Functions
by: Li, Yuwen, et al.
Published: (2025)

Linearization of ReLU Activation Function for Neural Network-Embedded Optimization: Optimal Day-Ahead Energy Scheduling
by: Zhao, Cunzhi, et al.
Published: (2023)

Two-Dimensional Deep ReLU CNN Approximation for Korobov Functions: A Constructive Approach
by: Fang, Qin, et al.
Published: (2025)

Beyond ReLU: Chebyshev-DQN for Enhanced Deep Q-Networks
by: Yazdannik, Saman, et al.
Published: (2025)

ZC-Swish: Stabilizing Deep BN-Free Networks for Edge and Micro-Batch Applications
by: Basak, Suvinava
Published: (2026)

Convexity in ReLU Neural Networks: beyond ICNNs?
by: Gagneux, Anne, et al.
Published: (2025)

Pathwise Explanation of ReLU Neural Networks
by: Lim, Seongwoo, et al.
Published: (2025)

TeLU Activation Function for Fast and Stable Deep Learning
by: Fernandez, Alfredo, et al.
Published: (2024)

Better NTK Conditioning: A Free Lunch from (ReLU) Nonlinear Activation in Wide Neural Networks
by: Liu, Chaoyue, et al.
Published: (2023)

Provable Accelerated Convergence of Nesterov's Momentum for Deep ReLU Neural Networks
by: Liao, Fangshuo, et al.
Published: (2023)

Activation-Descent Regularization for Input Optimization of ReLU Networks
by: Yu, Hongzhan, et al.
Published: (2024)

Stochastic Bandits with ReLU Neural Networks
by: Xu, Kan, et al.
Published: (2024)

On Space Folds of ReLU Neural Networks
by: Lewandowski, Michal, et al.
Published: (2025)

On the Local Complexity of Linear Regions in Deep ReLU Networks
by: Patel, Niket, et al.
Published: (2024)

Implicit Hypersurface Approximation Capacity in Deep ReLU Networks
by: Vallin, Jonatan, et al.
Published: (2024)

ReLU Networks as Random Functions: Their Distribution in Probability Space
by: Chaudhari, Shreyas, et al.
Published: (2025)

Hamiltonian Monte Carlo on ReLU Neural Networks is Inefficient
by: Dinh, Vu C., et al.
Published: (2024)

Topological Expressivity of ReLU Neural Networks
by: Ergen, Ekin, et al.
Published: (2023)

ReLU$^2$ Wins: Discovering Efficient Activation Functions for Sparse LLMs
by: Zhang, Zhengyan, et al.
Published: (2024)

Geometry-induced Regularization in Deep ReLU Neural Networks
by: Bona-Pellissier, Joachim, et al.
Published: (2024)

Random ReLU Neural Networks as Non-Gaussian Processes
by: Parhi, Rahul, et al.
Published: (2024)

Universal Consistency of Wide and Deep ReLU Neural Networks and Minimax Optimal Convergence Rates for Kolmogorov-Donoho Optimal Function Classes
by: Ko, Hyunouk, et al.
Published: (2024)

Stability and Performance Analysis of Discrete-Time ReLU Recurrent Neural Networks
by: Noori, Sahel Vahedi, et al.
Published: (2024)

Why ReLU? A Bit-Model Dichotomy for Deep Network Training
by: Doron-Arad, Ilan, et al.
Published: (2026)

Dense ReLU Neural Networks for Temporal-spatial Model
by: Padilla, Carlos Misael Madrid, et al.
Published: (2024)

On the Expressiveness of Rational ReLU Neural Networks With Bounded Depth
by: Averkov, Gennadiy, et al.
Published: (2025)