PPS-Ctrl: Controllable Sim-to-Real Translation for Colonoscopy Depth Estimation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912343916544000 |
|---|---|
| author | Xiong, Xinqi Beltran, Andrea Dunn Choi, Jun Myeong Niethammer, Marc Sengupta, Roni |
| author_facet | Xiong, Xinqi Beltran, Andrea Dunn Choi, Jun Myeong Niethammer, Marc Sengupta, Roni |
| contents | Accurate depth estimation enhances endoscopy navigation and diagnostics, but obtaining ground-truth depth in clinical settings is challenging. Synthetic datasets are often used for training, yet the domain gap limits generalization to real data. We propose a novel image-to-image translation framework that preserves structure while generating realistic textures from clinical data. Our key innovation integrates Stable Diffusion with ControlNet, conditioned on a latent representation extracted from a Per-Pixel Shading (PPS) map. PPS captures surface lighting effects, providing a stronger structural constraint than depth maps. Experiments show our approach produces more realistic translations and improves depth estimation over GAN-based MI-CycleGAN. Our code is publicly accessible at https://github.com/anaxqx/PPS-Ctrl. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_17067 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PPS-Ctrl: Controllable Sim-to-Real Translation for Colonoscopy Depth Estimation Xiong, Xinqi Beltran, Andrea Dunn Choi, Jun Myeong Niethammer, Marc Sengupta, Roni Computer Vision and Pattern Recognition Accurate depth estimation enhances endoscopy navigation and diagnostics, but obtaining ground-truth depth in clinical settings is challenging. Synthetic datasets are often used for training, yet the domain gap limits generalization to real data. We propose a novel image-to-image translation framework that preserves structure while generating realistic textures from clinical data. Our key innovation integrates Stable Diffusion with ControlNet, conditioned on a latent representation extracted from a Per-Pixel Shading (PPS) map. PPS captures surface lighting effects, providing a stronger structural constraint than depth maps. Experiments show our approach produces more realistic translations and improves depth estimation over GAN-based MI-CycleGAN. Our code is publicly accessible at https://github.com/anaxqx/PPS-Ctrl. |
| title | PPS-Ctrl: Controllable Sim-to-Real Translation for Colonoscopy Depth Estimation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.17067 |