Structure-preserving Image Translation for Depth Estimation in Colonoscopy Video
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909290835476480 |
|---|---|
| author | Wang, Shuxian Paruchuri, Akshay Zhang, Zhaoxi McGill, Sarah Sengupta, Roni |
| author_facet | Wang, Shuxian Paruchuri, Akshay Zhang, Zhaoxi McGill, Sarah Sengupta, Roni |
| contents | Monocular depth estimation in colonoscopy video aims to overcome the unusual lighting properties of the colonoscopic environment. One of the major challenges in this area is the domain gap between annotated but unrealistic synthetic data and unannotated but realistic clinical data. Previous attempts to bridge this domain gap directly target the depth estimation task itself. We propose a general pipeline of structure-preserving synthetic-to-real (sim2real) image translation (producing a modified version of the input image) to retain depth geometry through the translation process. This allows us to generate large quantities of realistic-looking synthetic images for supervised depth estimation with improved generalization to the clinical domain. We also propose a dataset of hand-picked sequences from clinical colonoscopies to improve the image translation process. We demonstrate the simultaneous realism of the translated images and preservation of depth maps via the performance of downstream depth estimation on various datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_10153 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Structure-preserving Image Translation for Depth Estimation in Colonoscopy Video Wang, Shuxian Paruchuri, Akshay Zhang, Zhaoxi McGill, Sarah Sengupta, Roni Computer Vision and Pattern Recognition Monocular depth estimation in colonoscopy video aims to overcome the unusual lighting properties of the colonoscopic environment. One of the major challenges in this area is the domain gap between annotated but unrealistic synthetic data and unannotated but realistic clinical data. Previous attempts to bridge this domain gap directly target the depth estimation task itself. We propose a general pipeline of structure-preserving synthetic-to-real (sim2real) image translation (producing a modified version of the input image) to retain depth geometry through the translation process. This allows us to generate large quantities of realistic-looking synthetic images for supervised depth estimation with improved generalization to the clinical domain. We also propose a dataset of hand-picked sequences from clinical colonoscopies to improve the image translation process. We demonstrate the simultaneous realism of the translated images and preservation of depth maps via the performance of downstream depth estimation on various datasets. |
| title | Structure-preserving Image Translation for Depth Estimation in Colonoscopy Video |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2408.10153 |