Event-to-Video Conversion for Overhead Object Detection
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914673887019008 |
|---|---|
| author | Hannan, Darryl Arnab, Ragib Parpart, Gavin Kenyon, Garrett T. Kim, Edward Watkins, Yijing |
| author_facet | Hannan, Darryl Arnab, Ragib Parpart, Gavin Kenyon, Garrett T. Kim, Edward Watkins, Yijing |
| contents | Collecting overhead imagery using an event camera is desirable due to the energy efficiency of the image sensor compared to standard cameras. However, event cameras complicate downstream image processing, especially for complex tasks such as object detection. In this paper, we investigate the viability of event streams for overhead object detection. We demonstrate that across a number of standard modeling approaches, there is a significant gap in performance between dense event representations and corresponding RGB frames. We establish that this gap is, in part, due to a lack of overlap between the event representations and the pre-training data used to initialize the weights of the object detectors. Then, we apply event-to-video conversion models that convert event streams into gray-scale video to close this gap. We demonstrate that this approach results in a large performance increase, outperforming even event-specific object detection techniques on our overhead target task. These results suggest that better alignment between event representations and existing large pre-trained models may result in greater short-term performance gains compared to end-to-end event-specific architectural improvements. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_06805 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Event-to-Video Conversion for Overhead Object Detection Hannan, Darryl Arnab, Ragib Parpart, Gavin Kenyon, Garrett T. Kim, Edward Watkins, Yijing Computer Vision and Pattern Recognition Collecting overhead imagery using an event camera is desirable due to the energy efficiency of the image sensor compared to standard cameras. However, event cameras complicate downstream image processing, especially for complex tasks such as object detection. In this paper, we investigate the viability of event streams for overhead object detection. We demonstrate that across a number of standard modeling approaches, there is a significant gap in performance between dense event representations and corresponding RGB frames. We establish that this gap is, in part, due to a lack of overlap between the event representations and the pre-training data used to initialize the weights of the object detectors. Then, we apply event-to-video conversion models that convert event streams into gray-scale video to close this gap. We demonstrate that this approach results in a large performance increase, outperforming even event-specific object detection techniques on our overhead target task. These results suggest that better alignment between event representations and existing large pre-trained models may result in greater short-term performance gains compared to end-to-end event-specific architectural improvements. |
| title | Event-to-Video Conversion for Overhead Object Detection |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2402.06805 |