Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.18672 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913869207699456 |
|---|---|
| author | Heiman, Alice Zhang, Xiaoman Chen, Emma Kim, Sung Eun Rajpurkar, Pranav |
| author_facet | Heiman, Alice Zhang, Xiaoman Chen, Emma Kim, Sung Eun Rajpurkar, Pranav |
| contents | Medical vision-language models often struggle with generating accurate quantitative measurements in radiology reports, leading to hallucinations that undermine clinical reliability. We introduce FactCheXcker, a modular framework that de-hallucinates radiology report measurements by leveraging an improved query-code-update paradigm. Specifically, FactCheXcker employs specialized modules and the code generation capabilities of large language models to solve measurement queries generated based on the original report. After extracting measurable findings, the results are incorporated into an updated report. We evaluate FactCheXcker on endotracheal tube placement, which accounts for an average of 78% of report measurements, using the MIMIC-CXR dataset and 11 medical report-generation models. Our results show that FactCheXcker significantly reduces hallucinations, improves measurement precision, and maintains the quality of the original reports. Specifically, FactCheXcker improves the performance of 10/11 models and achieves an average improvement of 135.0% in reducing measurement hallucinations measured by mean absolute error. Code is available at https://github.com/rajpurkarlab/FactCheXcker. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_18672 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | FactCheXcker: Mitigating Measurement Hallucinations in Chest X-ray Report Generation Models Heiman, Alice Zhang, Xiaoman Chen, Emma Kim, Sung Eun Rajpurkar, Pranav Computer Vision and Pattern Recognition Medical vision-language models often struggle with generating accurate quantitative measurements in radiology reports, leading to hallucinations that undermine clinical reliability. We introduce FactCheXcker, a modular framework that de-hallucinates radiology report measurements by leveraging an improved query-code-update paradigm. Specifically, FactCheXcker employs specialized modules and the code generation capabilities of large language models to solve measurement queries generated based on the original report. After extracting measurable findings, the results are incorporated into an updated report. We evaluate FactCheXcker on endotracheal tube placement, which accounts for an average of 78% of report measurements, using the MIMIC-CXR dataset and 11 medical report-generation models. Our results show that FactCheXcker significantly reduces hallucinations, improves measurement precision, and maintains the quality of the original reports. Specifically, FactCheXcker improves the performance of 10/11 models and achieves an average improvement of 135.0% in reducing measurement hallucinations measured by mean absolute error. Code is available at https://github.com/rajpurkarlab/FactCheXcker. |
| title | FactCheXcker: Mitigating Measurement Hallucinations in Chest X-ray Report Generation Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2411.18672 |