General Vision Encoder Features as Guidance in Medical Image Registration
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913435226210304 |
|---|---|
| author | Kögl, Fryderyk Reithmeir, Anna Sideri-Lampretsa, Vasiliki Machado, Ines Braren, Rickmer Rückert, Daniel Schnabel, Julia A. Zimmer, Veronika A. |
| author_facet | Kögl, Fryderyk Reithmeir, Anna Sideri-Lampretsa, Vasiliki Machado, Ines Braren, Rickmer Rückert, Daniel Schnabel, Julia A. Zimmer, Veronika A. |
| contents | General vision encoders like DINOv2 and SAM have recently transformed computer vision. Even though they are trained on natural images, such encoder models have excelled in medical imaging, e.g., in classification, segmentation, and registration. However, no in-depth comparison of different state-of-the-art general vision encoders for medical registration is available. In this work, we investigate how well general vision encoder features can be used in the dissimilarity metrics for medical image registration. We explore two encoders that were trained on natural images as well as one that was fine-tuned on medical data. We apply the features within the well-established B-spline FFD registration framework. In extensive experiments on cardiac cine MRI data, we find that using features as additional guidance for conventional metrics improves the registration quality. The code is available at github.com/compai-lab/2024-miccai-koegl. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_13311 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | General Vision Encoder Features as Guidance in Medical Image Registration Kögl, Fryderyk Reithmeir, Anna Sideri-Lampretsa, Vasiliki Machado, Ines Braren, Rickmer Rückert, Daniel Schnabel, Julia A. Zimmer, Veronika A. Computer Vision and Pattern Recognition General vision encoders like DINOv2 and SAM have recently transformed computer vision. Even though they are trained on natural images, such encoder models have excelled in medical imaging, e.g., in classification, segmentation, and registration. However, no in-depth comparison of different state-of-the-art general vision encoders for medical registration is available. In this work, we investigate how well general vision encoder features can be used in the dissimilarity metrics for medical image registration. We explore two encoders that were trained on natural images as well as one that was fine-tuned on medical data. We apply the features within the well-established B-spline FFD registration framework. In extensive experiments on cardiac cine MRI data, we find that using features as additional guidance for conventional metrics improves the registration quality. The code is available at github.com/compai-lab/2024-miccai-koegl. |
| title | General Vision Encoder Features as Guidance in Medical Image Registration |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.13311 |