GS-EVT: Cross-Modal Event Camera Tracking based on Gaussian Splatting
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912049620058112 |
|---|---|
| author | Liu, Tao Yuan, Runze Ju, Yi'ang Xu, Xun Yang, Jiaqi Meng, Xiangting Lagorce, Xavier Kneip, Laurent |
| author_facet | Liu, Tao Yuan, Runze Ju, Yi'ang Xu, Xun Yang, Jiaqi Meng, Xiangting Lagorce, Xavier Kneip, Laurent |
| contents | Reliable self-localization is a foundational skill for many intelligent mobile platforms. This paper explores the use of event cameras for motion tracking thereby providing a solution with inherent robustness under difficult dynamics and illumination. In order to circumvent the challenge of event camera-based mapping, the solution is framed in a cross-modal way. It tracks a map representation that comes directly from frame-based cameras. Specifically, the proposed method operates on top of gaussian splatting, a state-of-the-art representation that permits highly efficient and realistic novel view synthesis. The key of our approach consists of a novel pose parametrization that uses a reference pose plus first order dynamics for local differential image rendering. The latter is then compared against images of integrated events in a staggered coarse-to-fine optimization scheme. As demonstrated by our results, the realistic view rendering ability of gaussian splatting leads to stable and accurate tracking across a variety of both publicly available and newly recorded data sequences. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_19228 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | GS-EVT: Cross-Modal Event Camera Tracking based on Gaussian Splatting Liu, Tao Yuan, Runze Ju, Yi'ang Xu, Xun Yang, Jiaqi Meng, Xiangting Lagorce, Xavier Kneip, Laurent Computer Vision and Pattern Recognition Reliable self-localization is a foundational skill for many intelligent mobile platforms. This paper explores the use of event cameras for motion tracking thereby providing a solution with inherent robustness under difficult dynamics and illumination. In order to circumvent the challenge of event camera-based mapping, the solution is framed in a cross-modal way. It tracks a map representation that comes directly from frame-based cameras. Specifically, the proposed method operates on top of gaussian splatting, a state-of-the-art representation that permits highly efficient and realistic novel view synthesis. The key of our approach consists of a novel pose parametrization that uses a reference pose plus first order dynamics for local differential image rendering. The latter is then compared against images of integrated events in a staggered coarse-to-fine optimization scheme. As demonstrated by our results, the realistic view rendering ability of gaussian splatting leads to stable and accurate tracking across a variety of both publicly available and newly recorded data sequences. |
| title | GS-EVT: Cross-Modal Event Camera Tracking based on Gaussian Splatting |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2409.19228 |