Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2401.11123 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914947570597888 |
|---|---|
| author | Yang, Haoxiang Yuan, Chengguo Zhu, Yabin Chen, Lan Wang, Xiao Wang, Futian |
| author_facet | Yang, Haoxiang Yuan, Chengguo Zhu, Yabin Chen, Lan Wang, Xiao Wang, Futian |
| contents | The mainstream human activity recognition (HAR) algorithms are developed based on RGB cameras, which are easily influenced by low-quality images (e.g., low illumination, motion blur). Meanwhile, the privacy protection issue caused by ultra-high definition (HD) RGB cameras aroused more and more people's attention. Inspired by the success of event cameras which perform better on high dynamic range, no motion blur, and low energy consumption, we propose to recognize human actions based on the event stream. We propose a lightweight uncertainty-aware information propagation based Mobile-Former network for efficient pattern recognition, which aggregates the MobileNet and Transformer network effectively. Specifically, we first embed the event images using a stem network into feature representations, then, feed them into uncertainty-aware Mobile-Former blocks for local and global feature learning and fusion. Finally, the features from MobileNet and Transformer branches are concatenated for pattern recognition. Extensive experiments on multiple event-based recognition datasets fully validated the effectiveness of our model. The source code of this work will be released at https://github.com/Event-AHU/Uncertainty_aware_MobileFormer. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_11123 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Uncertainty-aware Bridge based Mobile-Former Network for Event-based Pattern Recognition Yang, Haoxiang Yuan, Chengguo Zhu, Yabin Chen, Lan Wang, Xiao Wang, Futian Computer Vision and Pattern Recognition The mainstream human activity recognition (HAR) algorithms are developed based on RGB cameras, which are easily influenced by low-quality images (e.g., low illumination, motion blur). Meanwhile, the privacy protection issue caused by ultra-high definition (HD) RGB cameras aroused more and more people's attention. Inspired by the success of event cameras which perform better on high dynamic range, no motion blur, and low energy consumption, we propose to recognize human actions based on the event stream. We propose a lightweight uncertainty-aware information propagation based Mobile-Former network for efficient pattern recognition, which aggregates the MobileNet and Transformer network effectively. Specifically, we first embed the event images using a stem network into feature representations, then, feed them into uncertainty-aware Mobile-Former blocks for local and global feature learning and fusion. Finally, the features from MobileNet and Transformer branches are concatenated for pattern recognition. Extensive experiments on multiple event-based recognition datasets fully validated the effectiveness of our model. The source code of this work will be released at https://github.com/Event-AHU/Uncertainty_aware_MobileFormer. |
| title | Uncertainty-aware Bridge based Mobile-Former Network for Event-based Pattern Recognition |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2401.11123 |