Towards Interpretable Radiology Report Generation via Concept Bottlenecks using a Multi-Agentic RAG
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912352135282688 |
|---|---|
| author | Alam, Hasan Md Tusfiqur Srivastav, Devansh Kadir, Md Abdul Sonntag, Daniel |
| author_facet | Alam, Hasan Md Tusfiqur Srivastav, Devansh Kadir, Md Abdul Sonntag, Daniel |
| contents | Deep learning has advanced medical image classification, but interpretability challenges hinder its clinical adoption. This study enhances interpretability in Chest X-ray (CXR) classification by using concept bottleneck models (CBMs) and a multi-agent Retrieval-Augmented Generation (RAG) system for report generation. By modeling relationships between visual features and clinical concepts, we create interpretable concept vectors that guide a multi-agent RAG system to generate radiology reports, enhancing clinical relevance, explainability, and transparency. Evaluation of the generated reports using an LLM-as-a-judge confirmed the interpretability and clinical utility of our model's outputs. On the COVID-QU dataset, our model achieved 81% classification accuracy and demonstrated robust report generation performance, with five key metrics ranging between 84% and 90%. This interpretable multi-agent framework bridges the gap between high-performance AI and the explainability required for reliable AI-driven CXR analysis in clinical settings. Our code is available at https://github.com/tifat58/IRR-with-CBM-RAG.git. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_16086 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Towards Interpretable Radiology Report Generation via Concept Bottlenecks using a Multi-Agentic RAG Alam, Hasan Md Tusfiqur Srivastav, Devansh Kadir, Md Abdul Sonntag, Daniel Information Retrieval Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Image and Video Processing Deep learning has advanced medical image classification, but interpretability challenges hinder its clinical adoption. This study enhances interpretability in Chest X-ray (CXR) classification by using concept bottleneck models (CBMs) and a multi-agent Retrieval-Augmented Generation (RAG) system for report generation. By modeling relationships between visual features and clinical concepts, we create interpretable concept vectors that guide a multi-agent RAG system to generate radiology reports, enhancing clinical relevance, explainability, and transparency. Evaluation of the generated reports using an LLM-as-a-judge confirmed the interpretability and clinical utility of our model's outputs. On the COVID-QU dataset, our model achieved 81% classification accuracy and demonstrated robust report generation performance, with five key metrics ranging between 84% and 90%. This interpretable multi-agent framework bridges the gap between high-performance AI and the explainability required for reliable AI-driven CXR analysis in clinical settings. Our code is available at https://github.com/tifat58/IRR-with-CBM-RAG.git. |
| title | Towards Interpretable Radiology Report Generation via Concept Bottlenecks using a Multi-Agentic RAG |
| topic | Information Retrieval Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Image and Video Processing |
| url | https://arxiv.org/abs/2412.16086 |