Evaluating Automated Radiology Report Quality through Fine-Grained Phrasal Grounding of Clinical Findings
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908374599204864 |
|---|---|
| author | Mahmood, Razi Yan, Pingkun Reyes, Diego Machado Wang, Ge Kalra, Mannudeep K. Kaviani, Parisa Wu, Joy T. Syeda-Mahmood, Tanveer |
| author_facet | Mahmood, Razi Yan, Pingkun Reyes, Diego Machado Wang, Ge Kalra, Mannudeep K. Kaviani, Parisa Wu, Joy T. Syeda-Mahmood, Tanveer |
| contents | Several evaluation metrics have been developed recently to automatically assess the quality of generative AI reports for chest radiographs based only on textual information using lexical, semantic, or clinical named entity recognition methods. In this paper, we develop a new method of report quality evaluation by first extracting fine-grained finding patterns capturing the location, laterality, and severity of a large number of clinical findings. We then performed phrasal grounding to localize their associated anatomical regions on chest radiograph images. The textual and visual measures are then combined to rate the quality of the generated reports. We present results that compare this evaluation metric with other textual metrics on a gold standard dataset derived from the MIMIC collection and show its robustness and sensitivity to factual errors. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_01031 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Evaluating Automated Radiology Report Quality through Fine-Grained Phrasal Grounding of Clinical Findings Mahmood, Razi Yan, Pingkun Reyes, Diego Machado Wang, Ge Kalra, Mannudeep K. Kaviani, Parisa Wu, Joy T. Syeda-Mahmood, Tanveer Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition Several evaluation metrics have been developed recently to automatically assess the quality of generative AI reports for chest radiographs based only on textual information using lexical, semantic, or clinical named entity recognition methods. In this paper, we develop a new method of report quality evaluation by first extracting fine-grained finding patterns capturing the location, laterality, and severity of a large number of clinical findings. We then performed phrasal grounding to localize their associated anatomical regions on chest radiograph images. The textual and visual measures are then combined to rate the quality of the generated reports. We present results that compare this evaluation metric with other textual metrics on a gold standard dataset derived from the MIMIC collection and show its robustness and sensitivity to factual errors. |
| title | Evaluating Automated Radiology Report Quality through Fine-Grained Phrasal Grounding of Clinical Findings |
| topic | Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.01031 |