Free-Moving Object Reconstruction and Pose Estimation with Virtual Camera
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913345715568640 |
|---|---|
| author | Shi, Haixin Hu, Yinlin Koguciuk, Daniel Lin, Juan-Ting Salzmann, Mathieu Ferstl, David |
| author_facet | Shi, Haixin Hu, Yinlin Koguciuk, Daniel Lin, Juan-Ting Salzmann, Mathieu Ferstl, David |
| contents | We propose an approach for reconstructing free-moving object from a monocular RGB video. Most existing methods either assume scene prior, hand pose prior, object category pose prior, or rely on local optimization with multiple sequence segments. We propose a method that allows free interaction with the object in front of a moving camera without relying on any prior, and optimizes the sequence globally without any segments. We progressively optimize the object shape and pose simultaneously based on an implicit neural representation. A key aspect of our method is a virtual camera system that reduces the search space of the optimization significantly. We evaluate our method on the standard HO3D dataset and a collection of egocentric RGB sequences captured with a head-mounted device. We demonstrate that our approach outperforms most methods significantly, and is on par with recent techniques that assume prior information. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_05858 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Free-Moving Object Reconstruction and Pose Estimation with Virtual Camera Shi, Haixin Hu, Yinlin Koguciuk, Daniel Lin, Juan-Ting Salzmann, Mathieu Ferstl, David Computer Vision and Pattern Recognition Artificial Intelligence Graphics Robotics We propose an approach for reconstructing free-moving object from a monocular RGB video. Most existing methods either assume scene prior, hand pose prior, object category pose prior, or rely on local optimization with multiple sequence segments. We propose a method that allows free interaction with the object in front of a moving camera without relying on any prior, and optimizes the sequence globally without any segments. We progressively optimize the object shape and pose simultaneously based on an implicit neural representation. A key aspect of our method is a virtual camera system that reduces the search space of the optimization significantly. We evaluate our method on the standard HO3D dataset and a collection of egocentric RGB sequences captured with a head-mounted device. We demonstrate that our approach outperforms most methods significantly, and is on par with recent techniques that assume prior information. |
| title | Free-Moving Object Reconstruction and Pose Estimation with Virtual Camera |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Graphics Robotics |
| url | https://arxiv.org/abs/2405.05858 |