RGB-T Object Detection via Group Shuffled Multi-receptive Attention and Multi-modal Supervision
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910462389518336 |
|---|---|
| author | Wang, Jinzhong Tian, Xuetao Dai, Shun Zhuo, Tao Zeng, Haorui Liu, Hongjuan Liu, Jiaqi Zhang, Xiuwei Zhang, Yanning |
| author_facet | Wang, Jinzhong Tian, Xuetao Dai, Shun Zhuo, Tao Zeng, Haorui Liu, Hongjuan Liu, Jiaqi Zhang, Xiuwei Zhang, Yanning |
| contents | Multispectral object detection, utilizing both visible (RGB) and thermal infrared (T) modals, has garnered significant attention for its robust performance across diverse weather and lighting conditions. However, effectively exploiting the complementarity between RGB-T modals while maintaining efficiency remains a critical challenge. In this paper, a very simple Group Shuffled Multi-receptive Attention (GSMA) module is proposed to extract and combine multi-scale RGB and thermal features. Then, the extracted multi-modal features are directly integrated with a multi-level path aggregation neck, which significantly improves the fusion effect and efficiency. Meanwhile, multi-modal object detection often adopts union annotations for both modals. This kind of supervision is not sufficient and unfair, since objects observed in one modal may not be seen in the other modal. To solve this issue, Multi-modal Supervision (MS) is proposed to sufficiently supervise RGB-T object detection. Comprehensive experiments on two challenging benchmarks, KAIST and DroneVehicle, demonstrate the proposed model achieves the state-of-the-art accuracy while maintaining competitive efficiency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_18955 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | RGB-T Object Detection via Group Shuffled Multi-receptive Attention and Multi-modal Supervision Wang, Jinzhong Tian, Xuetao Dai, Shun Zhuo, Tao Zeng, Haorui Liu, Hongjuan Liu, Jiaqi Zhang, Xiuwei Zhang, Yanning Computer Vision and Pattern Recognition Multispectral object detection, utilizing both visible (RGB) and thermal infrared (T) modals, has garnered significant attention for its robust performance across diverse weather and lighting conditions. However, effectively exploiting the complementarity between RGB-T modals while maintaining efficiency remains a critical challenge. In this paper, a very simple Group Shuffled Multi-receptive Attention (GSMA) module is proposed to extract and combine multi-scale RGB and thermal features. Then, the extracted multi-modal features are directly integrated with a multi-level path aggregation neck, which significantly improves the fusion effect and efficiency. Meanwhile, multi-modal object detection often adopts union annotations for both modals. This kind of supervision is not sufficient and unfair, since objects observed in one modal may not be seen in the other modal. To solve this issue, Multi-modal Supervision (MS) is proposed to sufficiently supervise RGB-T object detection. Comprehensive experiments on two challenging benchmarks, KAIST and DroneVehicle, demonstrate the proposed model achieves the state-of-the-art accuracy while maintaining competitive efficiency. |
| title | RGB-T Object Detection via Group Shuffled Multi-receptive Attention and Multi-modal Supervision |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.18955 |