Unsupervised Domain Transfer with Conditional Invertible Neural Networks
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909074434555904 |
|---|---|
| author | Dreher, Kris K. Ayala, Leonardo Schellenberg, Melanie Hübner, Marco Nölke, Jan-Hinrich Adler, Tim J. Seidlitz, Silvia Sellner, Jan Studier-Fischer, Alexander Gröhl, Janek Nickel, Felix Köthe, Ullrich Seitel, Alexander Maier-Hein, Lena |
| author_facet | Dreher, Kris K. Ayala, Leonardo Schellenberg, Melanie Hübner, Marco Nölke, Jan-Hinrich Adler, Tim J. Seidlitz, Silvia Sellner, Jan Studier-Fischer, Alexander Gröhl, Janek Nickel, Felix Köthe, Ullrich Seitel, Alexander Maier-Hein, Lena |
| contents | Synthetic medical image generation has evolved as a key technique for neural network training and validation. A core challenge, however, remains in the domain gap between simulations and real data. While deep learning-based domain transfer using Cycle Generative Adversarial Networks and similar architectures has led to substantial progress in the field, there are use cases in which state-of-the-art approaches still fail to generate training images that produce convincing results on relevant downstream tasks. Here, we address this issue with a domain transfer approach based on conditional invertible neural networks (cINNs). As a particular advantage, our method inherently guarantees cycle consistency through its invertible architecture, and network training can efficiently be conducted with maximum likelihood training. To showcase our method's generic applicability, we apply it to two spectral imaging modalities at different scales, namely hyperspectral imaging (pixel-level) and photoacoustic tomography (image-level). According to comprehensive experiments, our method enables the generation of realistic spectral data and outperforms the state of the art on two downstream classification tasks (binary and multi-class). cINN-based domain transfer could thus evolve as an important method for realistic synthetic data generation in the field of spectral imaging and beyond. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2303_10191 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Unsupervised Domain Transfer with Conditional Invertible Neural Networks Dreher, Kris K. Ayala, Leonardo Schellenberg, Melanie Hübner, Marco Nölke, Jan-Hinrich Adler, Tim J. Seidlitz, Silvia Sellner, Jan Studier-Fischer, Alexander Gröhl, Janek Nickel, Felix Köthe, Ullrich Seitel, Alexander Maier-Hein, Lena Image and Video Processing Computer Vision and Pattern Recognition Machine Learning Synthetic medical image generation has evolved as a key technique for neural network training and validation. A core challenge, however, remains in the domain gap between simulations and real data. While deep learning-based domain transfer using Cycle Generative Adversarial Networks and similar architectures has led to substantial progress in the field, there are use cases in which state-of-the-art approaches still fail to generate training images that produce convincing results on relevant downstream tasks. Here, we address this issue with a domain transfer approach based on conditional invertible neural networks (cINNs). As a particular advantage, our method inherently guarantees cycle consistency through its invertible architecture, and network training can efficiently be conducted with maximum likelihood training. To showcase our method's generic applicability, we apply it to two spectral imaging modalities at different scales, namely hyperspectral imaging (pixel-level) and photoacoustic tomography (image-level). According to comprehensive experiments, our method enables the generation of realistic spectral data and outperforms the state of the art on two downstream classification tasks (binary and multi-class). cINN-based domain transfer could thus evolve as an important method for realistic synthetic data generation in the field of spectral imaging and beyond. |
| title | Unsupervised Domain Transfer with Conditional Invertible Neural Networks |
| topic | Image and Video Processing Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2303.10191 |