Spike-NeRF: Neural Radiance Field Based On Spike Camera
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910381485588480 |
|---|---|
| author | Guo, Yijia Bai, Yuanxi Hu, Liwen Liu, Mianzhi Guo, Ziyi Ma, Lei Huang, Tiejun |
| author_facet | Guo, Yijia Bai, Yuanxi Hu, Liwen Liu, Mianzhi Guo, Ziyi Ma, Lei Huang, Tiejun |
| contents | As a neuromorphic sensor with high temporal resolution, spike cameras offer notable advantages over traditional cameras in high-speed vision applications such as high-speed optical estimation, depth estimation, and object tracking. Inspired by the success of the spike camera, we proposed Spike-NeRF, the first Neural Radiance Field derived from spike data, to achieve 3D reconstruction and novel viewpoint synthesis of high-speed scenes. Instead of the multi-view images at the same time of NeRF, the inputs of Spike-NeRF are continuous spike streams captured by a moving spike camera in a very short time. To reconstruct a correct and stable 3D scene from high-frequency but unstable spike data, we devised spike masks along with a distinctive loss function. We evaluate our method qualitatively and numerically on several challenging synthetic scenes generated by blender with the spike camera simulator. Our results demonstrate that Spike-NeRF produces more visually appealing results than the existing methods and the baseline we proposed in high-speed scenes. Our code and data will be released soon. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_16410 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Spike-NeRF: Neural Radiance Field Based On Spike Camera Guo, Yijia Bai, Yuanxi Hu, Liwen Liu, Mianzhi Guo, Ziyi Ma, Lei Huang, Tiejun Computer Vision and Pattern Recognition As a neuromorphic sensor with high temporal resolution, spike cameras offer notable advantages over traditional cameras in high-speed vision applications such as high-speed optical estimation, depth estimation, and object tracking. Inspired by the success of the spike camera, we proposed Spike-NeRF, the first Neural Radiance Field derived from spike data, to achieve 3D reconstruction and novel viewpoint synthesis of high-speed scenes. Instead of the multi-view images at the same time of NeRF, the inputs of Spike-NeRF are continuous spike streams captured by a moving spike camera in a very short time. To reconstruct a correct and stable 3D scene from high-frequency but unstable spike data, we devised spike masks along with a distinctive loss function. We evaluate our method qualitatively and numerically on several challenging synthetic scenes generated by blender with the spike camera simulator. Our results demonstrate that Spike-NeRF produces more visually appealing results than the existing methods and the baseline we proposed in high-speed scenes. Our code and data will be released soon. |
| title | Spike-NeRF: Neural Radiance Field Based On Spike Camera |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.16410 |