RTFormer: Re-parameter TSBN Spiking Transformer
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916295818084352 |
|---|---|
| author | Wang, Hongzhi Liang, Xiubo Li, Mengjian Zhang, Tao |
| author_facet | Wang, Hongzhi Liang, Xiubo Li, Mengjian Zhang, Tao |
| contents | The Spiking Neural Networks (SNNs), renowned for their bio-inspired operational mechanism and energy efficiency, mirror the human brain's neural activity. Yet, SNNs face challenges in balancing energy efficiency with the computational demands of advanced tasks. Our research introduces the RTFormer, a novel architecture that embeds Re-parameterized Temporal Sliding Batch Normalization (TSBN) within the Spiking Transformer framework. This innovation optimizes energy usage during inference while ensuring robust computational performance. The crux of RTFormer lies in its integration of reparameterized convolutions and TSBN, achieving an equilibrium between computational prowess and energy conservation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_14180 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | RTFormer: Re-parameter TSBN Spiking Transformer Wang, Hongzhi Liang, Xiubo Li, Mengjian Zhang, Tao Neural and Evolutionary Computing The Spiking Neural Networks (SNNs), renowned for their bio-inspired operational mechanism and energy efficiency, mirror the human brain's neural activity. Yet, SNNs face challenges in balancing energy efficiency with the computational demands of advanced tasks. Our research introduces the RTFormer, a novel architecture that embeds Re-parameterized Temporal Sliding Batch Normalization (TSBN) within the Spiking Transformer framework. This innovation optimizes energy usage during inference while ensuring robust computational performance. The crux of RTFormer lies in its integration of reparameterized convolutions and TSBN, achieving an equilibrium between computational prowess and energy conservation. |
| title | RTFormer: Re-parameter TSBN Spiking Transformer |
| topic | Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2406.14180 |