Cover Image

Hysteresis Activation Function for Efficient Inference

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kimhi, Moshe, Kashani, Idan, Mendelson, Avi, Baskin, Chaim
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language Neural and Evolutionary Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

EvolKV: Evolutionary KV Cache Compression for LLM Inference
by: Yu, Bohan, et al.
Published: (2025)

GRASP: GRouped Activation Shared Parameterization for Parameter-Efficient Fine-Tuning and Robust Inference of Transformers
by: Bal, Malyaban, et al.
Published: (2025)

Classy Ensemble: A Novel Ensemble Algorithm for Classification
by: Sipper, Moshe
Published: (2023)

HAT: Hardware-Aware Transformers for Efficient Natural Language Processing
by: Wang, Hanrui, et al.
Published: (2020)

Deriving Activation Functions Using Integration
by: Huang, Allen Hao, et al.
Published: (2024)

Expanded Gating Ranges Improve Activation Functions
by: Huang, Allen Hao
Published: (2024)

Topology-Aware Activation Functions in Neural Networks
by: Snopov, Pavel, et al.
Published: (2025)

Activation Functions for "A Feedforward Unitary Equivariant Neural Network"
by: Ma, Pui-Wai
Published: (2024)

Adaptive Activation Functions for Predictive Modeling with Sparse Experimental Data
by: Pourkamali-Anaraki, Farhad, et al.
Published: (2024)

AMED: Automatic Mixed-Precision Quantization for Edge Devices
by: Kimhi, Moshe, et al.
Published: (2022)

APALU: A Trainable, Adaptive Activation Function for Deep Learning Networks
by: Subramanian, Barathi, et al.
Published: (2024)

A More Accurate Approximation of Activation Function with Few Spikes Neurons
by: Jeong, Dayena, et al.
Published: (2024)

ActNAS : Generating Efficient YOLO Models using Activation NAS
by: Sah, Sudhakar, et al.
Published: (2024)

Deep Learning-Based Operators for Evolutionary Algorithms
by: Shem-Tov, Eliad, et al.
Published: (2024)

Vector Policy Optimization: Training for Diversity Improves Test-Time Search
by: Bahlous-Boldi, Ryan, et al.
Published: (2026)

Learnable Activation Functions in Physics-Informed Neural Networks for Solving Partial Differential Equations
by: Farea, Afrah, et al.
Published: (2024)

Evolving Multi-Channel Confidence-Aware Activation Functions for Missing Data with Channel Propagation
by: Sani, Naeem Shahabi, et al.
Published: (2026)

Efficient Deep Spiking Multi-Layer Perceptrons with Multiplication-Free Inference
by: Li, Boyan, et al.
Published: (2023)

Zorro: A Flexible and Differentiable Parametric Family of Activation Functions That Extends ReLU and GELU
by: Roodschild, Matias, et al.
Published: (2024)

Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Models
by: Dong, Peijie, et al.
Published: (2024)

SpikeLM: Towards General Spike-Driven Language Modeling via Elastic Bi-Spiking Mechanisms
by: Xing, Xingrun, et al.
Published: (2024)

Genetic Instruct: Scaling up Synthetic Generation of Coding Instructions for Large Language Models
by: Majumdar, Somshubra, et al.
Published: (2024)

Large Language Models for Tuning Evolution Strategies
by: Kramer, Oliver
Published: (2024)

On the Power of Convolution Augmented Transformer
by: Li, Mingchen, et al.
Published: (2024)

SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models
by: Shen, Shuaijie, et al.
Published: (2024)

Sorbet: A Neuromorphic Hardware-Compatible Transformer-Based Spiking Language Model
by: Tang, Kaiwen, et al.
Published: (2024)

An enhanced Teaching-Learning-Based Optimization (TLBO) with Grey Wolf Optimizer (GWO) for text feature selection and clustering
by: Azarshab, Mahsa, et al.
Published: (2024)

SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spiking
by: Xing, Xingrun, et al.
Published: (2024)

BrainTransformers: SNN-LLM
by: Tang, Zhengzheng, et al.
Published: (2024)

Improving Sequence-to-Sequence Models for Abstractive Text Summarization Using Meta Heuristic Approaches
by: Saxena, Aditya, et al.
Published: (2024)

B'MOJO: Hybrid State Space Realizations of Foundation Models with Eidetic and Fading Memory
by: Zancato, Luca, et al.
Published: (2024)

Neural Information Organizing and Processing -- Neural Machines
by: Petrila, Iosif Iulian
Published: (2024)

Decomposing Evolutionary Mixture-of-LoRA Architectures: The Routing Lever, the Lifecycle Penalty, and a Substrate-Conditional Boundary
by: Kumaresan, Ramchand
Published: (2026)

An In-depth Walkthrough on Evolution of Neural Machine Translation
by: Jagtap, Rohan, et al.
Published: (2020)

Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loop
by: Briesch, Martin, et al.
Published: (2023)

Pre-trained Language Models Learn Remarkably Accurate Representations of Numbers
by: Kadlčík, Marek, et al.
Published: (2025)

AP-BMM: Approximating Capability-Cost Pareto Sets of LLMs via Asynchronous Prior-Guided Bayesian Model Merging
by: Chen, Kesheng, et al.
Published: (2025)

A Hormone-inspired Emotion Layer for Transformer language models (HELT)
by: Reda, Eslam, et al.
Published: (2026)

SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention
by: Csordás, Róbert, et al.
Published: (2023)

EvoX: Meta-Evolution for Automated Discovery
by: Liu, Shu, et al.
Published: (2026)