Retinal IPA: Iterative KeyPoints Alignment for Multimodal Retinal Imaging
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910543227387904 |
|---|---|
| author | Wang, Jiacheng Li, Hao Hu, Dewei Xu, Rui Yao, Xing Tao, Yuankai K. Oguz, Ipek |
| author_facet | Wang, Jiacheng Li, Hao Hu, Dewei Xu, Rui Yao, Xing Tao, Yuankai K. Oguz, Ipek |
| contents | We propose a novel framework for retinal feature point alignment, designed for learning cross-modality features to enhance matching and registration across multi-modality retinal images. Our model draws on the success of previous learning-based feature detection and description methods. To better leverage unlabeled data and constrain the model to reproduce relevant keypoints, we integrate a keypoint-based segmentation task. It is trained in a self-supervised manner by enforcing segmentation consistency between different augmentations of the same image. By incorporating a keypoint augmented self-supervised layer, we achieve robust feature extraction across modalities. Extensive evaluation on two public datasets and one in-house dataset demonstrates significant improvements in performance for modality-agnostic retinal feature alignment. Our code and model weights are publicly available at \url{https://github.com/MedICL-VU/RetinaIPA}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_18362 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Retinal IPA: Iterative KeyPoints Alignment for Multimodal Retinal Imaging Wang, Jiacheng Li, Hao Hu, Dewei Xu, Rui Yao, Xing Tao, Yuankai K. Oguz, Ipek Image and Video Processing Computer Vision and Pattern Recognition Machine Learning We propose a novel framework for retinal feature point alignment, designed for learning cross-modality features to enhance matching and registration across multi-modality retinal images. Our model draws on the success of previous learning-based feature detection and description methods. To better leverage unlabeled data and constrain the model to reproduce relevant keypoints, we integrate a keypoint-based segmentation task. It is trained in a self-supervised manner by enforcing segmentation consistency between different augmentations of the same image. By incorporating a keypoint augmented self-supervised layer, we achieve robust feature extraction across modalities. Extensive evaluation on two public datasets and one in-house dataset demonstrates significant improvements in performance for modality-agnostic retinal feature alignment. Our code and model weights are publicly available at \url{https://github.com/MedICL-VU/RetinaIPA}. |
| title | Retinal IPA: Iterative KeyPoints Alignment for Multimodal Retinal Imaging |
| topic | Image and Video Processing Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2407.18362 |