Cover Image

Winner-Take-All Spiking Transformer for Language Modeling

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhou, Chenlin, Guo, Sihang, Wang, Jiaqi, Ma, Dongyang, Che, Kaiwei, Chen, Baiyu, Meng, Qingyan, Ma, Zhengyu, Tian, Yonghong
Format:	Preprint
Published:	2026
Subjects:	Neural and Evolutionary Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Adaptive Spiking Neurons for Vision and Language Modeling
by: Zhou, Chenlin, et al.
Published: (2026)

BiSpikCLM: A Spiking Language Model integrating Softmax-Free Spiking Attention and Spike-Aware Alignment Distillation
by: Guo, Sihang, et al.
Published: (2026)

Efficiently Training Time-to-First-Spike Spiking Neural Networks from Scratch
by: Che, Kaiwei, et al.
Published: (2024)

Spikingformer: A Key Foundation Model for Spiking Neural Networks
by: Zhou, Chenlin, et al.
Published: (2023)

A Self-Ensemble Inspired Approach for Effective Training of Binary-Weight Spiking Neural Networks
by: Meng, Qingyan, et al.
Published: (2025)

Temporal-adaptive Weight Quantization for Spiking Neural Networks
by: Zhang, Han, et al.
Published: (2025)

Spatial-Temporal Search for Spiking Neural Networks
by: Che, Kaiwei, et al.
Published: (2024)

Direct Training High-Performance Deep Spiking Neural Networks: A Review of Theories and Methods
by: Zhou, Chenlin, et al.
Published: (2024)

Scalable Dendritic Modeling Advances Expressive and Robust Deep Spiking Neural Networks
by: Huang, Yifan, et al.
Published: (2024)

One-Timestep is Enough: Achieving High-performance ANN-to-SNN Conversion via Scale-and-Fire Neurons
by: Chen, Qiuyang, et al.
Published: (2025)

Multiplication-Free Parallelizable Spiking Neurons with Efficient Spatio-Temporal Dynamics
by: Xue, Peng, et al.
Published: (2025)

Long-Range Feedback Spiking Network Captures Dynamic and Static Representations of the Visual Cortex under Movie Stimuli
by: Huang, Liwei, et al.
Published: (2023)

QKFormer: Hierarchical Spiking Transformer using Q-K Attention
by: Zhou, Chenlin, et al.
Published: (2024)

E2ATST: A Temporal-Spatial Optimized Energy-Efficient Architecture for Training Spiking Transformer
by: Ma, Yunhao, et al.
Published: (2025)

Core Placement Optimization of Many-core Brain-Inspired Near-Storage Systems for Spiking Neural Network Training
by: Zhu, Xueke, et al.
Published: (2024)

k-Winners-Take-All Ensemble Neural Network
by: Agarap, Abien Fred, et al.
Published: (2024)

Parallel Spiking Neurons with High Efficiency and Ability to Learn Long-term Dependencies
by: Fang, Wei, et al.
Published: (2023)

Spike-driven Large Language Model
by: Xu, Han, et al.
Published: (2026)

Noisy Spiking Actor Network for Exploration
by: Chen, Ding, et al.
Published: (2024)

Spiking Wavelet Transformer
by: Fang, Yuetong, et al.
Published: (2024)

SpikeZIP-TF: Conversion is All You Need for Transformer-based SNN
by: You, Kang, et al.
Published: (2024)

Spatio-Temporal Decoupled Learning for Spiking Neural Networks
by: Ma, Chenxiang, et al.
Published: (2025)

Deep Reinforcement Learning with Spiking Q-learning
by: Chen, Ding, et al.
Published: (2022)

Improving Low-Latency Learning Performance in Spiking Neural Networks via a Change-Perceptive Dendrite-Soma-Axon Neuron
by: Huang, Zeyu, et al.
Published: (2025)

Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips
by: Yao, Man, et al.
Published: (2024)

SpikePool: Event-driven Spiking Transformer with Pooling Attention
by: Lee, Donghyun, et al.
Published: (2025)

TC-LIF: A Two-Compartment Spiking Neuron Model for Long-Term Sequential Modelling
by: Zhang, Shimin, et al.
Published: (2023)

Efficient Online Learning for Networks of Two-Compartment Spiking Neurons
by: Yin, Yujia, et al.
Published: (2024)

Fully Spiking Actor Network with Intra-layer Connections for Reinforcement Learning
by: Chen, Ding, et al.
Published: (2024)

TEFormer: Structured Bidirectional Temporal Enhancement Modeling in Spiking Transformers
by: Shen, Sicheng, et al.
Published: (2026)

Online Pseudo-Zeroth-Order Training of Neuromorphic Spiking Neural Networks
by: Xiao, Mingqing, et al.
Published: (2024)

SpikingBERT: Distilling BERT to Train Spiking Language Models Using Implicit Differentiation
by: Bal, Malyaban, et al.
Published: (2023)

Exploring Extreme Quantization in Spiking Language Models
by: Bal, Malyaban, et al.
Published: (2024)

High-Performance Temporal Reversible Spiking Neural Networks with $O(L)$ Training Memory and $O(1)$ Inference Cost
by: Hu, JiaKui, et al.
Published: (2024)

Spiking Transformer with Spatial-Temporal Attention
by: Lee, Donghyun, et al.
Published: (2024)

Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks
by: Xiao, Mingqing, et al.
Published: (2024)

GP and LLMs for Program Synthesis: No Clear Winners
by: Hernandez, Jose Guadalupe, et al.
Published: (2025)

Time-Evolving Dynamical System for Learning Latent Representations of Mouse Visual Neural Activity
by: Huang, Liwei, et al.
Published: (2024)

Toward Relative Positional Encoding in Spiking Transformers
by: Lv, Changze, et al.
Published: (2025)

RTFormer: Re-parameter TSBN Spiking Transformer
by: Wang, Hongzhi, et al.
Published: (2024)