Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2407.08231 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929417615310848 |
|---|---|
| author | Liang, Jinxiu Yu, Bohan Yang, Yixin Han, Yiming Shi, Boxin |
| author_facet | Liang, Jinxiu Yu, Bohan Yang, Yixin Han, Yiming Shi, Boxin |
| contents | Event cameras, mimicking the human retina, capture brightness changes with unparalleled temporal resolution and dynamic range. Integrating events into intensities poses a highly ill-posed challenge, marred by initial condition ambiguities. Traditional regression-based deep learning methods fall short in perceptual quality, offering deterministic and often unrealistic reconstructions. In this paper, we introduce diffusion models to events-to-video reconstruction, achieving colorful, realistic, and perceptually superior video generation from achromatic events. Powered by the image generation ability and knowledge of pretrained diffusion models, the proposed method can achieve a better trade-off between the perception and distortion of the reconstructed frame compared to previous solutions. Extensive experiments on benchmark datasets demonstrate that our approach can produce diverse, realistic frames with faithfulness to the given events. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_08231 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | E2VIDiff: Perceptual Events-to-Video Reconstruction using Diffusion Priors Liang, Jinxiu Yu, Bohan Yang, Yixin Han, Yiming Shi, Boxin Computer Vision and Pattern Recognition Event cameras, mimicking the human retina, capture brightness changes with unparalleled temporal resolution and dynamic range. Integrating events into intensities poses a highly ill-posed challenge, marred by initial condition ambiguities. Traditional regression-based deep learning methods fall short in perceptual quality, offering deterministic and often unrealistic reconstructions. In this paper, we introduce diffusion models to events-to-video reconstruction, achieving colorful, realistic, and perceptually superior video generation from achromatic events. Powered by the image generation ability and knowledge of pretrained diffusion models, the proposed method can achieve a better trade-off between the perception and distortion of the reconstructed frame compared to previous solutions. Extensive experiments on benchmark datasets demonstrate that our approach can produce diverse, realistic frames with faithfulness to the given events. |
| title | E2VIDiff: Perceptual Events-to-Video Reconstruction using Diffusion Priors |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.08231 |