Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2305.12621 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913322772725760 |
|---|---|
| author | Sinha, Ashish Kawahara, Jeremy Pakzad, Arezou Abhishek, Kumar Ruthven, Matthieu Ghorbel, Enjie Kacem, Anis Aouada, Djamila Hamarneh, Ghassan |
| author_facet | Sinha, Ashish Kawahara, Jeremy Pakzad, Arezou Abhishek, Kumar Ruthven, Matthieu Ghorbel, Enjie Kacem, Anis Aouada, Djamila Hamarneh, Ghassan |
| contents | In recent years, deep learning (DL) has shown great potential in the field of dermatological image analysis. However, existing datasets in this domain have significant limitations, including a small number of image samples, limited disease conditions, insufficient annotations, and non-standardized image acquisitions. To address these shortcomings, we propose a novel framework called DermSynth3D. DermSynth3D blends skin disease patterns onto 3D textured meshes of human subjects using a differentiable renderer and generates 2D images from various camera viewpoints under chosen lighting conditions in diverse background scenes. Our method adheres to top-down rules that constrain the blending and rendering process to create 2D images with skin conditions that mimic in-the-wild acquisitions, ensuring more meaningful results. The framework generates photo-realistic 2D dermoscopy images and the corresponding dense annotations for semantic segmentation of the skin, skin conditions, body parts, bounding boxes around lesions, depth maps, and other 3D scene parameters, such as camera position and lighting conditions. DermSynth3D allows for the creation of custom datasets for various dermatology tasks. We demonstrate the effectiveness of data generated using DermSynth3D by training DL models on synthetic data and evaluating them on various dermatology tasks using real 2D dermatological images. We make our code publicly available at https://github.com/sfu-mial/DermSynth3D. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_12621 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | DermSynth3D: Synthesis of in-the-wild Annotated Dermatology Images Sinha, Ashish Kawahara, Jeremy Pakzad, Arezou Abhishek, Kumar Ruthven, Matthieu Ghorbel, Enjie Kacem, Anis Aouada, Djamila Hamarneh, Ghassan Image and Video Processing Computer Vision and Pattern Recognition Machine Learning In recent years, deep learning (DL) has shown great potential in the field of dermatological image analysis. However, existing datasets in this domain have significant limitations, including a small number of image samples, limited disease conditions, insufficient annotations, and non-standardized image acquisitions. To address these shortcomings, we propose a novel framework called DermSynth3D. DermSynth3D blends skin disease patterns onto 3D textured meshes of human subjects using a differentiable renderer and generates 2D images from various camera viewpoints under chosen lighting conditions in diverse background scenes. Our method adheres to top-down rules that constrain the blending and rendering process to create 2D images with skin conditions that mimic in-the-wild acquisitions, ensuring more meaningful results. The framework generates photo-realistic 2D dermoscopy images and the corresponding dense annotations for semantic segmentation of the skin, skin conditions, body parts, bounding boxes around lesions, depth maps, and other 3D scene parameters, such as camera position and lighting conditions. DermSynth3D allows for the creation of custom datasets for various dermatology tasks. We demonstrate the effectiveness of data generated using DermSynth3D by training DL models on synthetic data and evaluating them on various dermatology tasks using real 2D dermatological images. We make our code publicly available at https://github.com/sfu-mial/DermSynth3D. |
| title | DermSynth3D: Synthesis of in-the-wild Annotated Dermatology Images |
| topic | Image and Video Processing Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2305.12621 |