ViPE: Video Pose Engine for 3D Geometric Perception
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916899920543744 |
|---|---|
| author | Huang, Jiahui Zhou, Qunjie Rabeti, Hesam Korovko, Aleksandr Ling, Huan Ren, Xuanchi Shen, Tianchang Gao, Jun Slepichev, Dmitry Lin, Chen-Hsuan Ren, Jiawei Xie, Kevin Biswas, Joydeep Leal-Taixe, Laura Fidler, Sanja |
| author_facet | Huang, Jiahui Zhou, Qunjie Rabeti, Hesam Korovko, Aleksandr Ling, Huan Ren, Xuanchi Shen, Tianchang Gao, Jun Slepichev, Dmitry Lin, Chen-Hsuan Ren, Jiawei Xie, Kevin Biswas, Joydeep Leal-Taixe, Laura Fidler, Sanja |
| contents | Accurate 3D geometric perception is an important prerequisite for a wide range of spatial AI systems. While state-of-the-art methods depend on large-scale training data, acquiring consistent and precise 3D annotations from in-the-wild videos remains a key challenge. In this work, we introduce ViPE, a handy and versatile video processing engine designed to bridge this gap. ViPE efficiently estimates camera intrinsics, camera motion, and dense, near-metric depth maps from unconstrained raw videos. It is robust to diverse scenarios, including dynamic selfie videos, cinematic shots, or dashcams, and supports various camera models such as pinhole, wide-angle, and 360° panoramas. We have benchmarked ViPE on multiple benchmarks. Notably, it outperforms existing uncalibrated pose estimation baselines by 18%/50% on TUM/KITTI sequences, and runs at 3-5FPS on a single GPU for standard input resolutions. We use ViPE to annotate a large-scale collection of videos. This collection includes around 100K real-world internet videos, 1M high-quality AI-generated videos, and 2K panoramic videos, totaling approximately 96M frames -- all annotated with accurate camera poses and dense depth maps. We open-source ViPE and the annotated dataset with the hope of accelerating the development of spatial AI systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_10934 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ViPE: Video Pose Engine for 3D Geometric Perception Huang, Jiahui Zhou, Qunjie Rabeti, Hesam Korovko, Aleksandr Ling, Huan Ren, Xuanchi Shen, Tianchang Gao, Jun Slepichev, Dmitry Lin, Chen-Hsuan Ren, Jiawei Xie, Kevin Biswas, Joydeep Leal-Taixe, Laura Fidler, Sanja Computer Vision and Pattern Recognition Graphics Robotics Image and Video Processing Accurate 3D geometric perception is an important prerequisite for a wide range of spatial AI systems. While state-of-the-art methods depend on large-scale training data, acquiring consistent and precise 3D annotations from in-the-wild videos remains a key challenge. In this work, we introduce ViPE, a handy and versatile video processing engine designed to bridge this gap. ViPE efficiently estimates camera intrinsics, camera motion, and dense, near-metric depth maps from unconstrained raw videos. It is robust to diverse scenarios, including dynamic selfie videos, cinematic shots, or dashcams, and supports various camera models such as pinhole, wide-angle, and 360° panoramas. We have benchmarked ViPE on multiple benchmarks. Notably, it outperforms existing uncalibrated pose estimation baselines by 18%/50% on TUM/KITTI sequences, and runs at 3-5FPS on a single GPU for standard input resolutions. We use ViPE to annotate a large-scale collection of videos. This collection includes around 100K real-world internet videos, 1M high-quality AI-generated videos, and 2K panoramic videos, totaling approximately 96M frames -- all annotated with accurate camera poses and dense depth maps. We open-source ViPE and the annotated dataset with the hope of accelerating the development of spatial AI systems. |
| title | ViPE: Video Pose Engine for 3D Geometric Perception |
| topic | Computer Vision and Pattern Recognition Graphics Robotics Image and Video Processing |
| url | https://arxiv.org/abs/2508.10934 |