UniGeo: Taming Video Diffusion for Unified Consistent Geometry Estimation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916768935575552 |
|---|---|
| author | Sun, Yang-Tian Yu, Xin Huang, Zehuan Huang, Yi-Hua Guo, Yuan-Chen Yang, Ziyi Cao, Yan-Pei Qi, Xiaojuan |
| author_facet | Sun, Yang-Tian Yu, Xin Huang, Zehuan Huang, Yi-Hua Guo, Yuan-Chen Yang, Ziyi Cao, Yan-Pei Qi, Xiaojuan |
| contents | Recently, methods leveraging diffusion model priors to assist monocular geometric estimation (e.g., depth and normal) have gained significant attention due to their strong generalization ability. However, most existing works focus on estimating geometric properties within the camera coordinate system of individual video frames, neglecting the inherent ability of diffusion models to determine inter-frame correspondence. In this work, we demonstrate that, through appropriate design and fine-tuning, the intrinsic consistency of video generation models can be effectively harnessed for consistent geometric estimation. Specifically, we 1) select geometric attributes in the global coordinate system that share the same correspondence with video frames as the prediction targets, 2) introduce a novel and efficient conditioning method by reusing positional encodings, and 3) enhance performance through joint training on multiple geometric attributes that share the same correspondence. Our results achieve superior performance in predicting global geometric attributes in videos and can be directly applied to reconstruction tasks. Even when trained solely on static video data, our approach exhibits the potential to generalize to dynamic video scenes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_24521 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | UniGeo: Taming Video Diffusion for Unified Consistent Geometry Estimation Sun, Yang-Tian Yu, Xin Huang, Zehuan Huang, Yi-Hua Guo, Yuan-Chen Yang, Ziyi Cao, Yan-Pei Qi, Xiaojuan Computer Vision and Pattern Recognition Recently, methods leveraging diffusion model priors to assist monocular geometric estimation (e.g., depth and normal) have gained significant attention due to their strong generalization ability. However, most existing works focus on estimating geometric properties within the camera coordinate system of individual video frames, neglecting the inherent ability of diffusion models to determine inter-frame correspondence. In this work, we demonstrate that, through appropriate design and fine-tuning, the intrinsic consistency of video generation models can be effectively harnessed for consistent geometric estimation. Specifically, we 1) select geometric attributes in the global coordinate system that share the same correspondence with video frames as the prediction targets, 2) introduce a novel and efficient conditioning method by reusing positional encodings, and 3) enhance performance through joint training on multiple geometric attributes that share the same correspondence. Our results achieve superior performance in predicting global geometric attributes in videos and can be directly applied to reconstruction tasks. Even when trained solely on static video data, our approach exhibits the potential to generalize to dynamic video scenes. |
| title | UniGeo: Taming Video Diffusion for Unified Consistent Geometry Estimation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.24521 |