Imagen de Portada

SpikingMamba: Towards Energy-Efficient Large Language Models via Knowledge Distillation from Mamba

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Huang, Yulong, Tang, Jianxiong, Wang, Chao, Wang, Ziyi, Zhang, Jianguo, Lu, Zhichao, Cheng, Bojun, Leng, Luziwei
Formato:	Preprint
Publicado:	2025
Materias:	Neural and Evolutionary Computing
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

SPikE-SSM: A Sparse, Precise, and Efficient Spiking State Space Model for Long Sequences Learning
por: Zhong, Yan, et al.
Publicado: (2024)

Evolutionary Spiking Neural Networks: A Survey
por: Shen, Shuaijie, et al.
Publicado: (2024)

SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models
por: Shen, Shuaijie, et al.
Publicado: (2024)

Spatial-Temporal Search for Spiking Neural Networks
por: Che, Kaiwei, et al.
Publicado: (2024)

Efficient Deep Spiking Multi-Layer Perceptrons with Multiplication-Free Inference
por: Li, Boyan, et al.
Publicado: (2023)

Spiking and Event-driven Neuromorphic Mamba Models for Efficient Speech Recognition
por: Ahmed, Tauseef, et al.
Publicado: (2026)

Mamba-Spike: Enhancing the Mamba Architecture with a Spiking Front-End for Efficient Temporal Data Processing
por: Qin, Jiahao, et al.
Publicado: (2024)

MAR: Efficient Large Language Models via Module-aware Architecture Refinement
por: Cai, Junhong, et al.
Publicado: (2026)

CLIF: Complementary Leaky Integrate-and-Fire Neuron for Spiking Neural Networks
por: Huang, Yulong, et al.
Publicado: (2024)

Accurate and Efficient Event-based Semantic Segmentation Using Adaptive Spiking Encoder-Decoder Network
por: Zhang, Rui, et al.
Publicado: (2023)

PRF: Parallel Resonate and Fire Neuron for Long Sequence Learning in Spiking Neural Networks
por: Huang, Yulong, et al.
Publicado: (2024)

SpikeAtConv: An Integrated Spiking-Convolutional Attention Architecture for Energy-Efficient Neuromorphic Vision Processing
por: Liao, Wangdan, et al.
Publicado: (2024)

BKDSNN: Enhancing the Performance of Learning-based Spiking Neural Networks Training with Blurred Knowledge Distillation
por: Xu, Zekai, et al.
Publicado: (2024)

Biologically Plausible Learning via Bidirectional Spike-Based Distillation
por: Lv, Changze, et al.
Publicado: (2025)

SpikingBERT: Distilling BERT to Train Spiking Language Models Using Implicit Differentiation
por: Bal, Malyaban, et al.
Publicado: (2023)

Adaptively Pruned Spiking Neural Networks for Energy-Efficient Intracortical Neural Decoding
por: Rivelli, Francesca, et al.
Publicado: (2025)

Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model
por: Liu, Fei, et al.
Publicado: (2024)

Energy-Efficient Implementation of Spiking Recurrent Cells on FPGA
por: Harmeling, Pascal, et al.
Publicado: (2026)

Towards Automated Knowledge Transfer in Evolutionary Multitasking via Large Language Models
por: Lyu, Xuebin, et al.
Publicado: (2024)

SpikeVox: Towards Energy-Efficient Speech Therapy Framework with Spike-driven Generative Language Models
por: Putra, Rachmad Vidya Wicaksana, et al.
Publicado: (2025)

Adaptive Spiking Neurons for Vision and Language Modeling
por: Zhou, Chenlin, et al.
Publicado: (2026)

Mamba-PTQ: Outlier Channels in Recurrent Large Language Models
por: Pierro, Alessandro, et al.
Publicado: (2024)

SpikeLM: Towards General Spike-Driven Language Modeling via Elastic Bi-Spiking Mechanisms
por: Xing, Xingrun, et al.
Publicado: (2024)

Toward Relative Positional Encoding in Spiking Transformers
por: Lv, Changze, et al.
Publicado: (2025)

Towards free-response paradigm: a theory on decision-making in spiking neural networks
por: Zhu, Zhichao, et al.
Publicado: (2024)

Sparse Spike Encoding of Channel Responses for Energy Efficient Human Activity Recognition
por: Cicciarella, Eleonora, et al.
Publicado: (2026)

Threshold Modulation for Online Test-Time Adaptation of Spiking Neural Networks
por: Zhao, Kejie, et al.
Publicado: (2025)

Parallel Spiking Unit for Efficient Training of Spiking Neural Networks
por: Li, Yang, et al.
Publicado: (2024)

Evolving Interdependent Operators with Large Language Models for Multi-Objective Combinatorial Optimization
por: Qiu, Junhao, et al.
Publicado: (2026)

Stochastic Spiking Neural Networks with First-to-Spike Coding
por: Jiang, Yi, et al.
Publicado: (2024)

Efficient and Effective Time-Series Forecasting with Spiking Neural Networks
por: Lv, Changze, et al.
Publicado: (2024)

Efficiently Training Time-to-First-Spike Spiking Neural Networks from Scratch
por: Che, Kaiwei, et al.
Publicado: (2024)

MD-SNN: Membrane Potential-aware Distillation on Quantized Spiking Neural Network
por: Lee, Donghyun, et al.
Publicado: (2025)

Winner-Take-All Spiking Transformer for Language Modeling
por: Zhou, Chenlin, et al.
Publicado: (2026)

Dynamic Activation with Knowledge Distillation for Energy-Efficient Spiking NN Ensembles
por: Konstantaropoulos, Orestis, et al.
Publicado: (2025)

Spiking Synaptic Penalty: Appropriate Penalty Term for Energy-Efficient Spiking Neural Networks
por: Suetake, Kazuma, et al.
Publicado: (2023)

Gated Attention Coding for Training High-performance and Efficient Spiking Neural Networks
por: Qiu, Xuerui, et al.
Publicado: (2023)

Towards Efficient Deep Spiking Neural Networks Construction with Spiking Activity based Pruning
por: Li, Yaxin, et al.
Publicado: (2024)

Spike-driven Large Language Model
por: Xu, Han, et al.
Publicado: (2026)

ASRC-SNN: Adaptive Skip Recurrent Connection Spiking Neural Network
por: Xu, Shang, et al.
Publicado: (2025)