Cover Image

SpikingBERT: Distilling BERT to Train Spiking Language Models Using Implicit Differentiation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Bal, Malyaban, Sengupta, Abhronil
Format:	Preprint
Published:	2023
Subjects:	Neural and Evolutionary Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

P-SpikeSSM: Harnessing Probabilistic Spiking State Space Models for Long-Range Dependency Tasks
by: Bal, Malyaban, et al.
Published: (2024)

Exploring Extreme Quantization in Spiking Language Models
by: Bal, Malyaban, et al.
Published: (2024)

Scaling SNNs Trained Using Equilibrium Propagation to Convolutional Architectures
by: Lin, Jiaqi, et al.
Published: (2024)

GRASP: GRouped Activation Shared Parameterization for Parameter-Efficient Fine-Tuning and Robust Inference of Transformers
by: Bal, Malyaban, et al.
Published: (2025)

Spatio-Temporal Pruning for Compressed Spiking Large Language Models
by: Jiang, Yi, et al.
Published: (2025)

Benchmarking Spiking Neural Network Learning Methods with Varying Locality
by: Lin, Jiaqi, et al.
Published: (2024)

Stochastic Spiking Neural Networks with First-to-Spike Coding
by: Jiang, Yi, et al.
Published: (2024)

On the Adversarial Robustness of Spiking Neural Networks Trained by Local Learning
by: Lin, Jiaqi, et al.
Published: (2025)

Delving Deeper Into Astromorphic Transformers
by: Mia, Md Zesun Ahmed, et al.
Published: (2023)

RMAAT: Astrocyte-Inspired Memory Compression and Replay for Efficient Long-Context Transformers
by: Mia, Md Zesun Ahmed, et al.
Published: (2026)

Astrocyte Regulated Neuromorphic Central Pattern Generator Control of Legged Robotic Locomotion
by: Han, Zhuangyu, et al.
Published: (2023)

Neuromorphic Reinforcement Learning for Quadruped Locomotion Control on Uneven Terrain
by: Han, Zhuangyu, et al.
Published: (2026)

Parallel Spiking Unit for Efficient Training of Spiking Neural Networks
by: Li, Yang, et al.
Published: (2024)

Efficiently Training Time-to-First-Spike Spiking Neural Networks from Scratch
by: Che, Kaiwei, et al.
Published: (2024)

Hybrid Temporal-8-Bit Spike Coding for Spiking Neural Network Surrogate Training
by: Nhan, Luu Trong, et al.
Published: (2025)

Parallel Training in Spiking Neural Networks
by: Huang, Yanbin, et al.
Published: (2026)

BKDSNN: Enhancing the Performance of Learning-based Spiking Neural Networks Training with Blurred Knowledge Distillation
by: Xu, Zekai, et al.
Published: (2024)

Adaptive Spiking Neurons for Vision and Language Modeling
by: Zhou, Chenlin, et al.
Published: (2026)

SpikingMamba: Towards Energy-Efficient Large Language Models via Knowledge Distillation from Mamba
by: Huang, Yulong, et al.
Published: (2025)

Biologically Plausible Learning via Bidirectional Spike-Based Distillation
by: Lv, Changze, et al.
Published: (2025)

Winner-Take-All Spiking Transformer for Language Modeling
by: Zhou, Chenlin, et al.
Published: (2026)

Neuromorphic Cybersecurity with Semi-supervised Lifelong Learning
by: Mia, Md Zesun Ahmed, et al.
Published: (2025)

Spike Accumulation Forwarding for Effective Training of Spiking Neural Networks
by: Saiin, Ryuji, et al.
Published: (2023)

SpikePool: Event-driven Spiking Transformer with Pooling Attention
by: Lee, Donghyun, et al.
Published: (2025)

BiSpikCLM: A Spiking Language Model integrating Softmax-Free Spiking Attention and Spike-Aware Alignment Distillation
by: Guo, Sihang, et al.
Published: (2026)

Efficient Training of Spiking Neural Networks by Spike-aware Data Pruning
by: Ma, Chenxiang, et al.
Published: (2025)

MD-SNN: Membrane Potential-aware Distillation on Quantized Spiking Neural Network
by: Lee, Donghyun, et al.
Published: (2025)

Cannistraci-Hebb Training on Ultra-Sparse Spiking Neural Networks
by: Hua, Yuan, et al.
Published: (2025)

Full Integer Arithmetic Online Training for Spiking Neural Networks
by: Gomez, Ismael, et al.
Published: (2025)

TT-SNN: Tensor Train Decomposition for Efficient Spiking Neural Network Training
by: Lee, Donghyun, et al.
Published: (2024)

Stabilizing Spiking Neuron Training
by: Herranz-Celotti, Luca, et al.
Published: (2022)

To Spike or Not to Spike, that is the Question
by: Takaghaj, Sanaz Mahmoodi, et al.
Published: (2024)

Spiking Wavelet Transformer
by: Fang, Yuetong, et al.
Published: (2024)

Wafer2Spike: Spiking Neural Network for Wafer Map Pattern Classification
by: Mishra, Abhishek, et al.
Published: (2024)

Directly Training Temporal Spiking Neural Network with Sparse Surrogate Gradient
by: Li, Yang, et al.
Published: (2024)

Training Deep Normalization-Free Spiking Neural Networks with Lateral Inhibition
by: Liu, Peiyu, et al.
Published: (2025)

Spike-driven Large Language Model
by: Xu, Han, et al.
Published: (2026)

$SpikePack$: Enhanced Information Flow in Spiking Neural Networks with High Hardware Compatibility
by: Shen, Guobin, et al.
Published: (2025)

Spiking Brain Compression: Post-Training Second-order Compression for Spiking Neural Networks
by: Shi, Lianfeng, et al.
Published: (2025)

Expressivity of Spiking Neural Networks
by: Singh, Manjot, et al.
Published: (2023)