SDTrack: A Baseline for Event-based Tracking via Spiking Neural Networks
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916864348651520 |
|---|---|
| author | Shan, Yimeng Ren, Zhenbang Wu, Haodi Wei, Wenjie Zhu, Rui-Jie Wang, Shuai Zhang, Dehao Xiao, Yichen Zhang, Jieyuan Shi, Kexin Wang, Jingzhinan Eshraghian, Jason K. Qu, Haicheng Zhang, Malu |
| author_facet | Shan, Yimeng Ren, Zhenbang Wu, Haodi Wei, Wenjie Zhu, Rui-Jie Wang, Shuai Zhang, Dehao Xiao, Yichen Zhang, Jieyuan Shi, Kexin Wang, Jingzhinan Eshraghian, Jason K. Qu, Haicheng Zhang, Malu |
| contents | Event cameras provide superior temporal resolution, dynamic range, power efficiency, and pixel bandwidth. Spiking Neural Networks (SNNs) naturally complement event data through discrete spike signals, making them ideal for event-based tracking. However, current approaches that combine Artificial Neural Networks (ANNs) and SNNs, along with suboptimal architectures, compromise energy efficiency and limit tracking performance. To address these limitations, we propose the first Transformer-based spike-driven tracking pipeline. Our Global Trajectory Prompt (GTP) method effectively captures global trajectory information and aggregates it with event streams into event images to enhance spatiotemporal representation. We then introduce SDTrack, a Transformer-based spike-driven tracker comprising a Spiking MetaFormer backbone and a tracking head that directly predicts normalized coordinates using spike signals. The framework is end-to-end, does not require data augmentation or post-processing. Extensive experiments demonstrate that SDTrack achieves state-of-the-art performance while maintaining the lowest parameter count and energy consumption across multiple event-based tracking benchmarks, establishing a solid baseline for future research in the field of neuromorphic vision. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_08703 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SDTrack: A Baseline for Event-based Tracking via Spiking Neural Networks Shan, Yimeng Ren, Zhenbang Wu, Haodi Wei, Wenjie Zhu, Rui-Jie Wang, Shuai Zhang, Dehao Xiao, Yichen Zhang, Jieyuan Shi, Kexin Wang, Jingzhinan Eshraghian, Jason K. Qu, Haicheng Zhang, Malu Neural and Evolutionary Computing Computer Vision and Pattern Recognition Event cameras provide superior temporal resolution, dynamic range, power efficiency, and pixel bandwidth. Spiking Neural Networks (SNNs) naturally complement event data through discrete spike signals, making them ideal for event-based tracking. However, current approaches that combine Artificial Neural Networks (ANNs) and SNNs, along with suboptimal architectures, compromise energy efficiency and limit tracking performance. To address these limitations, we propose the first Transformer-based spike-driven tracking pipeline. Our Global Trajectory Prompt (GTP) method effectively captures global trajectory information and aggregates it with event streams into event images to enhance spatiotemporal representation. We then introduce SDTrack, a Transformer-based spike-driven tracker comprising a Spiking MetaFormer backbone and a tracking head that directly predicts normalized coordinates using spike signals. The framework is end-to-end, does not require data augmentation or post-processing. Extensive experiments demonstrate that SDTrack achieves state-of-the-art performance while maintaining the lowest parameter count and energy consumption across multiple event-based tracking benchmarks, establishing a solid baseline for future research in the field of neuromorphic vision. |
| title | SDTrack: A Baseline for Event-based Tracking via Spiking Neural Networks |
| topic | Neural and Evolutionary Computing Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.08703 |