Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.00981 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909674114121728 |
|---|---|
| author | Nugent, Jack Wu, Siyang Ma, Zeyu Han, Beining Parakh, Meenal Joshi, Abhishek Mei, Lingjie Raistrick, Alexander Li, Xinyuan Deng, Jia |
| author_facet | Nugent, Jack Wu, Siyang Ma, Zeyu Han, Beining Parakh, Meenal Joshi, Abhishek Mei, Lingjie Raistrick, Alexander Li, Xinyuan Deng, Jia |
| contents | Recent years have witnessed substantial progress on monocular depth estimation, particularly as measured by the success of large models on standard benchmarks. However, performance on standard benchmarks does not offer a complete assessment, because most evaluate accuracy but not robustness. In this work, we introduce PDE (Procedural Depth Evaluation), a new benchmark which enables systematic robustness evaluation. PDE uses procedural generation to create 3D scenes that test robustness to various controlled perturbations, including object, camera, material and lighting changes. Our analysis yields interesting findings on what perturbations are challenging for state-of-the-art depth models, which we hope will inform further research. Code and data are available at https://github.com/princeton-vl/proc-depth-eval. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_00981 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Evaluating Robustness of Monocular Depth Estimation with Procedural Scene Perturbations Nugent, Jack Wu, Siyang Ma, Zeyu Han, Beining Parakh, Meenal Joshi, Abhishek Mei, Lingjie Raistrick, Alexander Li, Xinyuan Deng, Jia Computer Vision and Pattern Recognition Recent years have witnessed substantial progress on monocular depth estimation, particularly as measured by the success of large models on standard benchmarks. However, performance on standard benchmarks does not offer a complete assessment, because most evaluate accuracy but not robustness. In this work, we introduce PDE (Procedural Depth Evaluation), a new benchmark which enables systematic robustness evaluation. PDE uses procedural generation to create 3D scenes that test robustness to various controlled perturbations, including object, camera, material and lighting changes. Our analysis yields interesting findings on what perturbations are challenging for state-of-the-art depth models, which we hope will inform further research. Code and data are available at https://github.com/princeton-vl/proc-depth-eval. |
| title | Evaluating Robustness of Monocular Depth Estimation with Procedural Scene Perturbations |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2507.00981 |