Physically Grounded Monocular Depth via Nanophotonic Wavefront Prompting
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909963529486336 |
|---|---|
| author | Li, Bingxuan Wu, Jiahao Xu, Yuan Zhu, Zezheng Zhang, Yunxiang Chen, Kenneth Liang, Yanqi Yu, Nanfang Sun, Qi |
| author_facet | Li, Bingxuan Wu, Jiahao Xu, Yuan Zhu, Zezheng Zhang, Yunxiang Chen, Kenneth Liang, Yanqi Yu, Nanfang Sun, Qi |
| contents | Depth foundation models offer strong learned priors for 3D perception but lack physical depth cues, leading to ambiguities in metric scale. We introduce a birefringent metalens -- a planar nanophotonic lens composed of subwavelength pixels for wavefront shaping with a thickness of 700 nm and a diameter of 3 mm -- to physically prompt depth foundation models. In a single monocular shot, our metalens physically embeds depth information into two polarized optical wavefronts, which we decode through a lightweight prompting and fine-tuning framework that aligns depth foundation models with the optical signals. To scale the training data, we develop a light wave propagation simulator that synthesizes metalens responses from RGB-D datasets, incorporating key physical factors to minimize the sim-to-real gap. Simulated and physical experiments with our fabricated titanium-dioxide metalens demonstrate accurate and consistent metric depth over state-of-the-art monocular depth estimators. The research demonstrates that nanophotonic wavefront formation offers a promising bridge for grounding depth foundation models in physical depth sensing. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_15770 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Physically Grounded Monocular Depth via Nanophotonic Wavefront Prompting Li, Bingxuan Wu, Jiahao Xu, Yuan Zhu, Zezheng Zhang, Yunxiang Chen, Kenneth Liang, Yanqi Yu, Nanfang Sun, Qi Optics Hardware Architecture Computer Vision and Pattern Recognition Depth foundation models offer strong learned priors for 3D perception but lack physical depth cues, leading to ambiguities in metric scale. We introduce a birefringent metalens -- a planar nanophotonic lens composed of subwavelength pixels for wavefront shaping with a thickness of 700 nm and a diameter of 3 mm -- to physically prompt depth foundation models. In a single monocular shot, our metalens physically embeds depth information into two polarized optical wavefronts, which we decode through a lightweight prompting and fine-tuning framework that aligns depth foundation models with the optical signals. To scale the training data, we develop a light wave propagation simulator that synthesizes metalens responses from RGB-D datasets, incorporating key physical factors to minimize the sim-to-real gap. Simulated and physical experiments with our fabricated titanium-dioxide metalens demonstrate accurate and consistent metric depth over state-of-the-art monocular depth estimators. The research demonstrates that nanophotonic wavefront formation offers a promising bridge for grounding depth foundation models in physical depth sensing. |
| title | Physically Grounded Monocular Depth via Nanophotonic Wavefront Prompting |
| topic | Optics Hardware Architecture Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.15770 |