Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.17358 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915300425859072 |
|---|---|
| author | Talegaonkar, Chinmay Suresh, Nikhil Gandudi Novack, Zachary Belhe, Yash Nagasamudra, Priyanka Antipa, Nicholas |
| author_facet | Talegaonkar, Chinmay Suresh, Nikhil Gandudi Novack, Zachary Belhe, Yash Nagasamudra, Priyanka Antipa, Nicholas |
| contents | Recent monocular metric depth estimation (MMDE) methods have made notable progress towards zero-shot generalization. However, they still exhibit a significant performance drop on out-of-distribution datasets. We address this limitation by injecting defocus blur cues at inference time into Marigold, a \textit{pre-trained} diffusion model for zero-shot, scale-invariant monocular depth estimation (MDE). Our method effectively turns Marigold into a metric depth predictor in a training-free manner. To incorporate defocus cues, we capture two images with a small and a large aperture from the same viewpoint. To recover metric depth, we then optimize the metric depth scaling parameters and the noise latents of Marigold at inference time using gradients from a loss function based on the defocus-blur image formation model. We compare our method against existing state-of-the-art zero-shot MMDE methods on a self-collected real dataset, showing quantitative and qualitative improvements. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_17358 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Repurposing Marigold for Zero-Shot Metric Depth Estimation via Defocus Blur Cues Talegaonkar, Chinmay Suresh, Nikhil Gandudi Novack, Zachary Belhe, Yash Nagasamudra, Priyanka Antipa, Nicholas Computer Vision and Pattern Recognition Recent monocular metric depth estimation (MMDE) methods have made notable progress towards zero-shot generalization. However, they still exhibit a significant performance drop on out-of-distribution datasets. We address this limitation by injecting defocus blur cues at inference time into Marigold, a \textit{pre-trained} diffusion model for zero-shot, scale-invariant monocular depth estimation (MDE). Our method effectively turns Marigold into a metric depth predictor in a training-free manner. To incorporate defocus cues, we capture two images with a small and a large aperture from the same viewpoint. To recover metric depth, we then optimize the metric depth scaling parameters and the noise latents of Marigold at inference time using gradients from a loss function based on the defocus-blur image formation model. We compare our method against existing state-of-the-art zero-shot MMDE methods on a self-collected real dataset, showing quantitative and qualitative improvements. |
| title | Repurposing Marigold for Zero-Shot Metric Depth Estimation via Defocus Blur Cues |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.17358 |