Cover Image

Three Decades of Activations: A Comprehensive Survey of 400 Activation Functions for Neural Networks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kunc, Vladimír, Kléma, Jiří
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Neural and Evolutionary Computing I.5.1
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Exploring the Relationship: Transformative Adaptive Activation Functions in Comparison to Other Activation Functions
by: Kunc, Vladimír
Published: (2024)

Timing-Based Backpropagation in Spiking Neural Networks Without Single-Spike Restrictions
by: Yamamoto, Kakei, et al.
Published: (2022)

Theoretical Analysis of the Advantage of Deepening Neural Networks
by: Esaki, Yasushi, et al.
Published: (2020)

Developing Convolutional Neural Networks using a Novel Lamarckian Co-Evolutionary Algorithm
by: Sharifi, Zaniar, et al.
Published: (2024)

Random Search as a Baseline for Sparse Neural Network Architecture Search
by: Farahani, Rezsa
Published: (2024)

Collapse or Preserve: Data-Dependent Temporal Aggregation for Spiking Neural Network Acceleration
by: Qin, Jiahao
Published: (2026)

K-Way Energy Probes for Metacognition Reduce to Softmax in Discriminative Predictive Coding Networks
by: Cacioli, Jon-Paul
Published: (2026)

Expanding continual few-shot learning benchmarks to include recognition of specific instances
by: Kowadlo, Gideon, et al.
Published: (2022)

Neuromorphic Parameter Estimation for Power Converter Health Monitoring Using Spiking Neural Networks
by: Baik, Hyeongmeen, et al.
Published: (2026)

Spiking Neural Network Architecture Search: A Survey
by: Svoboda, Kama, et al.
Published: (2025)

Multi-objective Genetic Programming with Multi-view Multi-level Feature for Enhanced Protein Secondary Structure Prediction
by: Qian, Yining, et al.
Published: (2026)

TabConv: Low-Computation CNN Inference via Table Lookups
by: Gupta, Neelesh, et al.
Published: (2024)

Enhanced Protein Intrinsic Disorder Prediction Through Dual-View Multiscale Features and Multi-objective Evolutionary Algorithm
by: Wang, Shaokuan, et al.
Published: (2026)

Distal Interference: Exploring the Limits of Model-Based Continual Learning
by: van Deventer, Heinrich, et al.
Published: (2024)

Neural Attention: A Novel Mechanism for Enhanced Expressive Power in Transformer Models
by: DiGiugno, Andrew, et al.
Published: (2025)

Pulse-Driven Neural Architecture: Learnable Oscillatory Dynamics for Robust Continuous-Time Sequence Processing
by: Sharma, Paras
Published: (2026)

Self-Distillation Learning Based on Temporal-Spatial Consistency for Spiking Neural Networks
by: Zuo, Lin, et al.
Published: (2024)

Deep Convolutional Autoencoder for Assessment of Drive-Cycle Anomalies in Connected Vehicle Sensor Data
by: Geglio, Anthony, et al.
Published: (2022)

Perforated Neural Networks for Keyword Spotting
by: Gopal, Vishy, et al.
Published: (2026)

Asynchronous Stochastic Gradient Descent with Decoupled Backpropagation and Layer-Wise Updates
by: Fokam, Cabrel Teguemne, et al.
Published: (2024)

mlx-snn: Spiking Neural Networks on Apple Silicon via MLX
by: Qin, Jiahao
Published: (2026)

Deep learning in a bilateral brain with hemispheric specialization
by: Rajagopalan, Chandramouli, et al.
Published: (2022)

Growing Cosine Unit: A Novel Oscillatory Activation Function That Can Speedup Training and Reduce Parameters in Convolutional Neural Networks
by: Noel, Mathew Mithra, et al.
Published: (2021)

Decoding Phone Pairs from MEG Signals Across Speech Modalities
by: de Zuazo, Xabier, et al.
Published: (2025)

Statistical Analysis of the Impact of Quaternion Components in Convolutional Neural Networks
by: Altamirano-Gómez, Gerardo, et al.
Published: (2024)

On the Convergence and Stability of Upside-Down Reinforcement Learning, Goal-Conditioned Supervised Learning, and Online Decision Transformers
by: Štrupl, Miroslav, et al.
Published: (2025)

Spyx: A Library for Just-In-Time Compiled Optimization of Spiking Neural Networks
by: Heckel, Kade M., et al.
Published: (2024)

Vector Symbolic Architectures answer Jackendoff's challenges for cognitive neuroscience
by: Gayler, Ross W.
Published: (2004)

Pendulum Model of Spiking Neurons
by: Bose, Joy
Published: (2025)

Eventprop training for efficient neuromorphic applications
by: Shoesmith, Thomas, et al.
Published: (2025)

The Hidden Linear Structure in Score-Based Models and its Application
by: Wang, Binxu, et al.
Published: (2023)

Dissecting Larval Zebrafish Hunting using Deep Reinforcement Learning Trained RNN Agents
by: Malik, Raaghav, et al.
Published: (2025)

Tensor Logic: The Language of AI
by: Domingos, Pedro
Published: (2025)

Topology-Aware Activation Functions in Neural Networks
by: Snopov, Pavel, et al.
Published: (2025)

Activation Functions for "A Feedforward Unitary Equivariant Neural Network"
by: Ma, Pui-Wai
Published: (2024)

Model Input-Output Configuration Search with Embedded Feature Selection for Sensor Time-series and Image Classification
by: Hoang, Anh T., et al.
Published: (2023)

Isotropic Activation Functions Enable Deindividuated Neurons and Adaptive Topologies
by: Bird, George
Published: (2026)

Spiking Sequence Machines and Transformers
by: Bose, Joy
Published: (2026)

Scalable Nested Optimization for Deep Learning
by: Lorraine, Jonathan
Published: (2024)

Energy-Efficient Neuromorphic Computing for Edge AI: A Framework with Adaptive Spiking Neural Networks and Hardware-Aware Optimization
by: Imanov, Olaf Yunus Laitinen, et al.
Published: (2026)