Region Masking to Accelerate Video Processing on Neuromorphic Hardware
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916658874941440 |
|---|---|
| author | Sarkar, Sreetama Shrestha, Sumit Bam Che, Yue Campos-Macias, Leobardo Datta, Gourav Beerel, Peter A. |
| author_facet | Sarkar, Sreetama Shrestha, Sumit Bam Che, Yue Campos-Macias, Leobardo Datta, Gourav Beerel, Peter A. |
| contents | The rapidly growing demand for on-chip edge intelligence on resource-constrained devices has motivated approaches to reduce energy and latency of deep learning models. Spiking neural networks (SNNs) have gained particular interest due to their promise to reduce energy consumption using event-based processing. We assert that while sigma-delta encoding in SNNs can take advantage of the temporal redundancy across video frames, they still involve a significant amount of redundant computations due to processing insignificant events. In this paper, we propose a region masking strategy that identifies regions of interest at the input of the SNN, thereby eliminating computation and data movement for events arising from unimportant regions. Our approach demonstrates that masking regions at the input not only significantly reduces the overall spiking activity of the network, but also provides significant improvement in throughput and latency. We apply region masking during video object detection on Loihi 2, demonstrating that masking approximately 60% of input regions can reduce energy-delay product by 1.65x over a baseline sigma-delta network, with a degradation in mAP@0.5 by 1.09%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_16775 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Region Masking to Accelerate Video Processing on Neuromorphic Hardware Sarkar, Sreetama Shrestha, Sumit Bam Che, Yue Campos-Macias, Leobardo Datta, Gourav Beerel, Peter A. Computer Vision and Pattern Recognition Neural and Evolutionary Computing Image and Video Processing The rapidly growing demand for on-chip edge intelligence on resource-constrained devices has motivated approaches to reduce energy and latency of deep learning models. Spiking neural networks (SNNs) have gained particular interest due to their promise to reduce energy consumption using event-based processing. We assert that while sigma-delta encoding in SNNs can take advantage of the temporal redundancy across video frames, they still involve a significant amount of redundant computations due to processing insignificant events. In this paper, we propose a region masking strategy that identifies regions of interest at the input of the SNN, thereby eliminating computation and data movement for events arising from unimportant regions. Our approach demonstrates that masking regions at the input not only significantly reduces the overall spiking activity of the network, but also provides significant improvement in throughput and latency. We apply region masking during video object detection on Loihi 2, demonstrating that masking approximately 60% of input regions can reduce energy-delay product by 1.65x over a baseline sigma-delta network, with a degradation in mAP@0.5 by 1.09%. |
| title | Region Masking to Accelerate Video Processing on Neuromorphic Hardware |
| topic | Computer Vision and Pattern Recognition Neural and Evolutionary Computing Image and Video Processing |
| url | https://arxiv.org/abs/2503.16775 |