Interactive Medical Image Analysis with Concept-based Similarity Reasoning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913729804763136 |
|---|---|
| author | Huy, Ta Duc Tran, Sen Kim Nguyen, Phan Tran, Nguyen Hoang Sam, Tran Bao Hengel, Anton van den Liao, Zhibin Verjans, Johan W. To, Minh-Son Phan, Vu Minh Hieu |
| author_facet | Huy, Ta Duc Tran, Sen Kim Nguyen, Phan Tran, Nguyen Hoang Sam, Tran Bao Hengel, Anton van den Liao, Zhibin Verjans, Johan W. To, Minh-Son Phan, Vu Minh Hieu |
| contents | The ability to interpret and intervene model decisions is important for the adoption of computer-aided diagnosis methods in clinical workflows. Recent concept-based methods link the model predictions with interpretable concepts and modify their activation scores to interact with the model. However, these concepts are at the image level, which hinders the model from pinpointing the exact patches the concepts are activated. Alternatively, prototype-based methods learn representations from training image patches and compare these with test image patches, using the similarity scores for final class prediction. However, interpreting the underlying concepts of these patches can be challenging and often necessitates post-hoc guesswork. To address this issue, this paper introduces the novel Concept-based Similarity Reasoning network (CSR), which offers (i) patch-level prototype with intrinsic concept interpretation, and (ii) spatial interactivity. First, the proposed CSR provides localized explanation by grounding prototypes of each concept on image regions. Second, our model introduces novel spatial-level interaction, allowing doctors to engage directly with specific image areas, making it an intuitive and transparent tool for medical imaging. CSR improves upon prior state-of-the-art interpretable methods by up to 4.5\% across three biomedical datasets. Our code is released at https://github.com/tadeephuy/InteractCSR. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_06873 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Interactive Medical Image Analysis with Concept-based Similarity Reasoning Huy, Ta Duc Tran, Sen Kim Nguyen, Phan Tran, Nguyen Hoang Sam, Tran Bao Hengel, Anton van den Liao, Zhibin Verjans, Johan W. To, Minh-Son Phan, Vu Minh Hieu Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning The ability to interpret and intervene model decisions is important for the adoption of computer-aided diagnosis methods in clinical workflows. Recent concept-based methods link the model predictions with interpretable concepts and modify their activation scores to interact with the model. However, these concepts are at the image level, which hinders the model from pinpointing the exact patches the concepts are activated. Alternatively, prototype-based methods learn representations from training image patches and compare these with test image patches, using the similarity scores for final class prediction. However, interpreting the underlying concepts of these patches can be challenging and often necessitates post-hoc guesswork. To address this issue, this paper introduces the novel Concept-based Similarity Reasoning network (CSR), which offers (i) patch-level prototype with intrinsic concept interpretation, and (ii) spatial interactivity. First, the proposed CSR provides localized explanation by grounding prototypes of each concept on image regions. Second, our model introduces novel spatial-level interaction, allowing doctors to engage directly with specific image areas, making it an intuitive and transparent tool for medical imaging. CSR improves upon prior state-of-the-art interpretable methods by up to 4.5\% across three biomedical datasets. Our code is released at https://github.com/tadeephuy/InteractCSR. |
| title | Interactive Medical Image Analysis with Concept-based Similarity Reasoning |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2503.06873 |