PathMR: Multimodal Visual Reasoning for Interpretable Pathology Diagnosis
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911127849402368 |
|---|---|
| author | Zhang, Ye Zhou, Yu Qi, Jingwen Zhang, Yongbing Puettmann, Simon Wichmann, Finn Ferreira, Larissa Pereira Sichward, Lara Keyl, Julius Hartmann, Sylvia Zhao, Shuo Wang, Hongxiao Xu, Xiaowei Chen, Jianxu |
| author_facet | Zhang, Ye Zhou, Yu Qi, Jingwen Zhang, Yongbing Puettmann, Simon Wichmann, Finn Ferreira, Larissa Pereira Sichward, Lara Keyl, Julius Hartmann, Sylvia Zhao, Shuo Wang, Hongxiao Xu, Xiaowei Chen, Jianxu |
| contents | Deep learning based automated pathological diagnosis has markedly improved diagnostic efficiency and reduced variability between observers, yet its clinical adoption remains limited by opaque model decisions and a lack of traceable rationale. To address this, recent multimodal visual reasoning architectures provide a unified framework that generates segmentation masks at the pixel level alongside semantically aligned textual explanations. By localizing lesion regions and producing expert style diagnostic narratives, these models deliver the transparent and interpretable insights necessary for dependable AI assisted pathology. Building on these advancements, we propose PathMR, a cell-level Multimodal visual Reasoning framework for Pathological image analysis. Given a pathological image and a textual query, PathMR generates expert-level diagnostic explanations while simultaneously predicting cell distribution patterns. To benchmark its performance, we evaluated our approach on the publicly available PathGen dataset as well as on our newly developed GADVR dataset. Extensive experiments on these two datasets demonstrate that PathMR consistently outperforms state-of-the-art visual reasoning methods in text generation quality, segmentation accuracy, and cross-modal alignment. These results highlight the potential of PathMR for improving interpretability in AI-driven pathological diagnosis. The code will be publicly available in https://github.com/zhangye-zoe/PathMR. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_20851 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PathMR: Multimodal Visual Reasoning for Interpretable Pathology Diagnosis Zhang, Ye Zhou, Yu Qi, Jingwen Zhang, Yongbing Puettmann, Simon Wichmann, Finn Ferreira, Larissa Pereira Sichward, Lara Keyl, Julius Hartmann, Sylvia Zhao, Shuo Wang, Hongxiao Xu, Xiaowei Chen, Jianxu Computer Vision and Pattern Recognition Deep learning based automated pathological diagnosis has markedly improved diagnostic efficiency and reduced variability between observers, yet its clinical adoption remains limited by opaque model decisions and a lack of traceable rationale. To address this, recent multimodal visual reasoning architectures provide a unified framework that generates segmentation masks at the pixel level alongside semantically aligned textual explanations. By localizing lesion regions and producing expert style diagnostic narratives, these models deliver the transparent and interpretable insights necessary for dependable AI assisted pathology. Building on these advancements, we propose PathMR, a cell-level Multimodal visual Reasoning framework for Pathological image analysis. Given a pathological image and a textual query, PathMR generates expert-level diagnostic explanations while simultaneously predicting cell distribution patterns. To benchmark its performance, we evaluated our approach on the publicly available PathGen dataset as well as on our newly developed GADVR dataset. Extensive experiments on these two datasets demonstrate that PathMR consistently outperforms state-of-the-art visual reasoning methods in text generation quality, segmentation accuracy, and cross-modal alignment. These results highlight the potential of PathMR for improving interpretability in AI-driven pathological diagnosis. The code will be publicly available in https://github.com/zhangye-zoe/PathMR. |
| title | PathMR: Multimodal Visual Reasoning for Interpretable Pathology Diagnosis |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2508.20851 |