Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.05933 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910887881736192 |
|---|---|
| author | Morelle, Olivier Bisten, Justus Wintergerst, Maximilian W. M. Finger, Robert P. Schultz, Thomas |
| author_facet | Morelle, Olivier Bisten, Justus Wintergerst, Maximilian W. M. Finger, Robert P. Schultz, Thomas |
| contents | Weakly supervised segmentation has the potential to greatly reduce the annotation effort for training segmentation models for small structures such as hyper-reflective foci (HRF) in optical coherence tomography (OCT). However, most weakly supervised methods either involve a strong downsampling of input images, or only achieve localization at a coarse resolution, both of which are unsatisfactory for small structures. We propose a novel framework that increases the spatial resolution of a traditional attention-based Multiple Instance Learning (MIL) approach by using Layer-wise Relevance Propagation (LRP) to prompt the Segment Anything Model (SAM~2), and increases recall with iterative inference. Moreover, we demonstrate that replacing MIL with a Compact Convolutional Transformer (CCT), which adds a positional encoding, and permits an exchange of information between different regions of the OCT image, leads to a further and substantial increase in segmentation accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_05933 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Weakly Supervised Segmentation of Hyper-Reflective Foci with Compact Convolutional Transformers and SAM2 Morelle, Olivier Bisten, Justus Wintergerst, Maximilian W. M. Finger, Robert P. Schultz, Thomas Computer Vision and Pattern Recognition Weakly supervised segmentation has the potential to greatly reduce the annotation effort for training segmentation models for small structures such as hyper-reflective foci (HRF) in optical coherence tomography (OCT). However, most weakly supervised methods either involve a strong downsampling of input images, or only achieve localization at a coarse resolution, both of which are unsatisfactory for small structures. We propose a novel framework that increases the spatial resolution of a traditional attention-based Multiple Instance Learning (MIL) approach by using Layer-wise Relevance Propagation (LRP) to prompt the Segment Anything Model (SAM~2), and increases recall with iterative inference. Moreover, we demonstrate that replacing MIL with a Compact Convolutional Transformer (CCT), which adds a positional encoding, and permits an exchange of information between different regions of the OCT image, leads to a further and substantial increase in segmentation accuracy. |
| title | Weakly Supervised Segmentation of Hyper-Reflective Foci with Compact Convolutional Transformers and SAM2 |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2501.05933 |