BeNeRF: Neural Radiance Fields from a Single Blurry Image and Event Stream
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917773676904448 |
|---|---|
| author | Li, Wenpu Wan, Pian Wang, Peng Li, Jinghang Zhou, Yi Liu, Peidong |
| author_facet | Li, Wenpu Wan, Pian Wang, Peng Li, Jinghang Zhou, Yi Liu, Peidong |
| contents | Neural implicit representation of visual scenes has attracted a lot of attention in recent research of computer vision and graphics. Most prior methods focus on how to reconstruct 3D scene representation from a set of images. In this work, we demonstrate the possibility to recover the neural radiance fields (NeRF) from a single blurry image and its corresponding event stream. We model the camera motion with a cubic B-Spline in SE(3) space. Both the blurry image and the brightness change within a time interval, can then be synthesized from the 3D scene representation given the 6-DoF poses interpolated from the cubic B-Spline. Our method can jointly learn both the implicit neural scene representation and recover the camera motion by minimizing the differences between the synthesized data and the real measurements without pre-computed camera poses from COLMAP. We evaluate the proposed method with both synthetic and real datasets. The experimental results demonstrate that we are able to render view-consistent latent sharp images from the learned NeRF and bring a blurry image alive in high quality. Code and data are available at https://github.com/wu-cvgl/BeNeRF. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_02174 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | BeNeRF: Neural Radiance Fields from a Single Blurry Image and Event Stream Li, Wenpu Wan, Pian Wang, Peng Li, Jinghang Zhou, Yi Liu, Peidong Computer Vision and Pattern Recognition Neural implicit representation of visual scenes has attracted a lot of attention in recent research of computer vision and graphics. Most prior methods focus on how to reconstruct 3D scene representation from a set of images. In this work, we demonstrate the possibility to recover the neural radiance fields (NeRF) from a single blurry image and its corresponding event stream. We model the camera motion with a cubic B-Spline in SE(3) space. Both the blurry image and the brightness change within a time interval, can then be synthesized from the 3D scene representation given the 6-DoF poses interpolated from the cubic B-Spline. Our method can jointly learn both the implicit neural scene representation and recover the camera motion by minimizing the differences between the synthesized data and the real measurements without pre-computed camera poses from COLMAP. We evaluate the proposed method with both synthetic and real datasets. The experimental results demonstrate that we are able to render view-consistent latent sharp images from the learned NeRF and bring a blurry image alive in high quality. Code and data are available at https://github.com/wu-cvgl/BeNeRF. |
| title | BeNeRF: Neural Radiance Fields from a Single Blurry Image and Event Stream |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.02174 |