HyperE2VID: Improving Event-Based Video Reconstruction via Hypernetworks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913266876284928 |
|---|---|
| author | Ercan, Burak Eker, Onur Saglam, Canberk Erdem, Aykut Erdem, Erkut |
| author_facet | Ercan, Burak Eker, Onur Saglam, Canberk Erdem, Aykut Erdem, Erkut |
| contents | Event-based cameras are becoming increasingly popular for their ability to capture high-speed motion with low latency and high dynamic range. However, generating videos from events remains challenging due to the highly sparse and varying nature of event data. To address this, in this study, we propose HyperE2VID, a dynamic neural network architecture for event-based video reconstruction. Our approach uses hypernetworks to generate per-pixel adaptive filters guided by a context fusion module that combines information from event voxel grids and previously reconstructed intensity images. We also employ a curriculum learning strategy to train the network more robustly. Our comprehensive experimental evaluations across various benchmark datasets reveal that HyperE2VID not only surpasses current state-of-the-art methods in terms of reconstruction quality but also achieves this with fewer parameters, reduced computational requirements, and accelerated inference times. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_06382 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | HyperE2VID: Improving Event-Based Video Reconstruction via Hypernetworks Ercan, Burak Eker, Onur Saglam, Canberk Erdem, Aykut Erdem, Erkut Computer Vision and Pattern Recognition Event-based cameras are becoming increasingly popular for their ability to capture high-speed motion with low latency and high dynamic range. However, generating videos from events remains challenging due to the highly sparse and varying nature of event data. To address this, in this study, we propose HyperE2VID, a dynamic neural network architecture for event-based video reconstruction. Our approach uses hypernetworks to generate per-pixel adaptive filters guided by a context fusion module that combines information from event voxel grids and previously reconstructed intensity images. We also employ a curriculum learning strategy to train the network more robustly. Our comprehensive experimental evaluations across various benchmark datasets reveal that HyperE2VID not only surpasses current state-of-the-art methods in terms of reconstruction quality but also achieves this with fewer parameters, reduced computational requirements, and accelerated inference times. |
| title | HyperE2VID: Improving Event-Based Video Reconstruction via Hypernetworks |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2305.06382 |