Cover Image

Activity Sparsity Complements Weight Sparsity for Efficient RNN Inference

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Mukherji, Rishav, Schöne, Mark, Nazeer, Khaleelulla Khan, Mayr, Christian, Subramoney, Anand
Format:	Preprint
Published:	2023
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Weight Sparsity Complements Activity Sparsity in Neuromorphic Language Models
by: Mukherji, Rishav, et al.
Published: (2024)

Language Modeling on a SpiNNaker 2 Neuromorphic Chip
by: Nazeer, Khaleelulla Khan, et al.
Published: (2023)

STREAM: A Universal State-Space Model for Sparse Geometric Data
by: Schöne, Mark, et al.
Published: (2024)

Asynchronous Stochastic Gradient Descent with Decoupled Backpropagation and Layer-Wise Updates
by: Fokam, Cabrel Teguemne, et al.
Published: (2024)

Scalable Event-by-event Processing of Neuromorphic Sensory Signals With Deep State-Space Models
by: Schöne, Mark, et al.
Published: (2024)

Understanding and Exploiting Weight Update Sparsity for Communication-Efficient Distributed RL
by: Miahi, Erfan, et al.
Published: (2026)

Polar Sparsity: High Throughput Batched LLM Inferencing with Scalable Contextual Sparsity
by: Shrestha, Susav, et al.
Published: (2025)

R-Sparse: Rank-Aware Activation Sparsity for Efficient LLM Inference
by: Zhang, Zhenyu, et al.
Published: (2025)

Sparsity Forcing: Reinforcing Token Sparsity of MLLMs
by: Chen, Feng, et al.
Published: (2025)

WiSparse: Boosting LLM Inference Efficiency with Weight-Aware Mixed Activation Sparsity
by: Chen, Lei, et al.
Published: (2026)

Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference
by: Tang, Jiaming, et al.
Published: (2024)

SVD Contextual Sparsity Predictors for Fast LLM Inference
by: Serbin, Georgii, et al.
Published: (2026)

HashAttention: Semantic Sparsity for Faster Inference
by: Desai, Aditya, et al.
Published: (2024)

Inference Time Context Sparsity: Illusion or Opportunity?
by: Joshi, Sahil, et al.
Published: (2026)

Efficient Deployment of Spiking Neural Networks on SpiNNaker2 for DVS Gesture Recognition Using Neuromorphic Intermediate Representation
by: Arfa, Sirine, et al.
Published: (2025)

SpiNNaker2: A Large-Scale Neuromorphic System for Event-Based and Asynchronous Machine Learning
by: Gonzalez, Hector A., et al.
Published: (2024)

LASERS: LAtent Space Encoding for Representations with Sparsity for Generative Modeling
by: Li, Xin, et al.
Published: (2024)

Exploring the limits of Hierarchical World Models in Reinforcement Learning
by: Schiewer, Robin, et al.
Published: (2024)

Federated Learning via Variational Bayesian Inference: Personalization, Sparsity and Clustering
by: Zhang, Xu, et al.
Published: (2023)

Mustafar: Promoting Unstructured Sparsity for KV Cache Pruning in LLM Inference
by: Joo, Donghyeon, et al.
Published: (2025)

GQSA: Group Quantization and Sparsity for Accelerating Large Language Model Inference
by: Zeng, Chao, et al.
Published: (2024)

Structured Sparsity and Weight-adaptive Pruning for Memory and Compute efficient Whisper models
by: Mudi, Prasenjit K, et al.
Published: (2025)

Improving Decision Sparsity
by: Sun, Yiyang, et al.
Published: (2024)

Homeostasis and Sparsity in Transformer
by: Kotyuzanskiy, Leonid, et al.
Published: (2024)

Improving MoE Compute Efficiency by Composing Weight and Data Sparsity
by: Kilian, Maciej, et al.
Published: (2026)

Weight Concentration Regularization for Improving Pruning Robustness Under High Sparsity
by: Yun, Vincent-Daniel, et al.
Published: (2025)

MoE-Infinity: Efficient MoE Inference on Personal Machines with Sparsity-Aware Expert Cache
by: Xue, Leyang, et al.
Published: (2024)

STS: Efficient Sparse Attention with Speculative Token Sparsity
by: Xu, Ceyu, et al.
Published: (2026)

An Efficient Training Algorithm for Models with Block-wise Sparsity
by: Zhu, Ding, et al.
Published: (2025)

Pivoting Factorization: A Compact Meta Low-Rank Representation of Sparsity for Efficient Inference in Large Language Models
by: Zhao, Jialin, et al.
Published: (2025)

Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity
by: Yin, Lu, et al.
Published: (2023)

Defending Membership Inference Attacks via Privacy-aware Sparsity Tuning
by: Hu, Qiang, et al.
Published: (2024)

Accelerating Transformer Inference and Training with 2:4 Activation Sparsity
by: Haziza, Daniel, et al.
Published: (2025)

Investigating Sparsity in Recurrent Neural Networks
by: Darji, Harshil
Published: (2024)

MOSAIC: Minimax-Optimal Sparsity-Adaptive Inference for Change Points in Dynamic Networks
by: Fan, Yingying, et al.
Published: (2025)

Leveraging Sparsity for Sample-Efficient Preference Learning: A Theoretical Perspective
by: Yao, Yunzhen, et al.
Published: (2025)

EcoSpa: Efficient Transformer Training with Coupled Sparsity
by: Xiao, Jinqi, et al.
Published: (2025)

Sparsity and Out-of-Distribution Generalization
by: Aaronson, Scott, et al.
Published: (2026)

Sparsity and Superposition in Mixture of Experts
by: Chaudhari, Marmik, et al.
Published: (2025)

Training event-based neural networks with exact gradients via Differentiable ODE Solving in JAX
by: König, Lukas, et al.
Published: (2026)