DepthFM: Fast Monocular Depth Estimation with Flow Matching
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916533734735872 |
|---|---|
| author | Gui, Ming Schusterbauer, Johannes Prestel, Ulrich Ma, Pingchuan Kotovenko, Dmytro Grebenkova, Olga Baumann, Stefan Andreas Hu, Vincent Tao Ommer, Björn |
| author_facet | Gui, Ming Schusterbauer, Johannes Prestel, Ulrich Ma, Pingchuan Kotovenko, Dmytro Grebenkova, Olga Baumann, Stefan Andreas Hu, Vincent Tao Ommer, Björn |
| contents | Current discriminative depth estimation methods often produce blurry artifacts, while generative approaches suffer from slow sampling due to curvatures in the noise-to-depth transport. Our method addresses these challenges by framing depth estimation as a direct transport between image and depth distributions. We are the first to explore flow matching in this field, and we demonstrate that its interpolation trajectories enhance both training and sampling efficiency while preserving high performance. While generative models typically require extensive training data, we mitigate this dependency by integrating external knowledge from a pre-trained image diffusion model, enabling effective transfer even across differing objectives. To further boost our model performance, we employ synthetic data and utilize image-depth pairs generated by a discriminative model on an in-the-wild image dataset. As a generative model, our model can reliably estimate depth confidence, which provides an additional advantage. Our approach achieves competitive zero-shot performance on standard benchmarks of complex natural scenes while improving sampling efficiency and only requiring minimal synthetic data for training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_13788 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DepthFM: Fast Monocular Depth Estimation with Flow Matching Gui, Ming Schusterbauer, Johannes Prestel, Ulrich Ma, Pingchuan Kotovenko, Dmytro Grebenkova, Olga Baumann, Stefan Andreas Hu, Vincent Tao Ommer, Björn Computer Vision and Pattern Recognition Current discriminative depth estimation methods often produce blurry artifacts, while generative approaches suffer from slow sampling due to curvatures in the noise-to-depth transport. Our method addresses these challenges by framing depth estimation as a direct transport between image and depth distributions. We are the first to explore flow matching in this field, and we demonstrate that its interpolation trajectories enhance both training and sampling efficiency while preserving high performance. While generative models typically require extensive training data, we mitigate this dependency by integrating external knowledge from a pre-trained image diffusion model, enabling effective transfer even across differing objectives. To further boost our model performance, we employ synthetic data and utilize image-depth pairs generated by a discriminative model on an in-the-wild image dataset. As a generative model, our model can reliably estimate depth confidence, which provides an additional advantage. Our approach achieves competitive zero-shot performance on standard benchmarks of complex natural scenes while improving sampling efficiency and only requiring minimal synthetic data for training. |
| title | DepthFM: Fast Monocular Depth Estimation with Flow Matching |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.13788 |