Align3R: Aligned Monocular Depth Estimation for Dynamic Videos
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917857862877184 |
|---|---|
| author | Lu, Jiahao Huang, Tianyu Li, Peng Dou, Zhiyang Lin, Cheng Cui, Zhiming Dong, Zhen Yeung, Sai-Kit Wang, Wenping Liu, Yuan |
| author_facet | Lu, Jiahao Huang, Tianyu Li, Peng Dou, Zhiyang Lin, Cheng Cui, Zhiming Dong, Zhen Yeung, Sai-Kit Wang, Wenping Liu, Yuan |
| contents | Recent developments in monocular depth estimation methods enable high-quality depth estimation of single-view images but fail to estimate consistent video depth across different frames. Recent works address this problem by applying a video diffusion model to generate video depth conditioned on the input video, which is training-expensive and can only produce scale-invariant depth values without camera poses. In this paper, we propose a novel video-depth estimation method called Align3R to estimate temporal consistent depth maps for a dynamic video. Our key idea is to utilize the recent DUSt3R model to align estimated monocular depth maps of different timesteps. First, we fine-tune the DUSt3R model with additional estimated monocular depth as inputs for the dynamic scenes. Then, we apply optimization to reconstruct both depth maps and camera poses. Extensive experiments demonstrate that Align3R estimates consistent video depth and camera poses for a monocular video with superior performance than baseline methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_03079 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Align3R: Aligned Monocular Depth Estimation for Dynamic Videos Lu, Jiahao Huang, Tianyu Li, Peng Dou, Zhiyang Lin, Cheng Cui, Zhiming Dong, Zhen Yeung, Sai-Kit Wang, Wenping Liu, Yuan Computer Vision and Pattern Recognition Recent developments in monocular depth estimation methods enable high-quality depth estimation of single-view images but fail to estimate consistent video depth across different frames. Recent works address this problem by applying a video diffusion model to generate video depth conditioned on the input video, which is training-expensive and can only produce scale-invariant depth values without camera poses. In this paper, we propose a novel video-depth estimation method called Align3R to estimate temporal consistent depth maps for a dynamic video. Our key idea is to utilize the recent DUSt3R model to align estimated monocular depth maps of different timesteps. First, we fine-tune the DUSt3R model with additional estimated monocular depth as inputs for the dynamic scenes. Then, we apply optimization to reconstruct both depth maps and camera poses. Extensive experiments demonstrate that Align3R estimates consistent video depth and camera poses for a monocular video with superior performance than baseline methods. |
| title | Align3R: Aligned Monocular Depth Estimation for Dynamic Videos |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.03079 |