Fact-Controlled Diagnosis of Hallucinations in Medical Text Summarization
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915315838877696 |
|---|---|
| author | BN, Suhas Shing, Han-Chin Xu, Lei Strong, Mitch Burnsky, Jon Ofor, Jessica Mason, Jordan R. Chen, Susan Srinivasan, Sundararajan Shivade, Chaitanya Moriarty, Jack Cohen, Joseph Paul |
| author_facet | BN, Suhas Shing, Han-Chin Xu, Lei Strong, Mitch Burnsky, Jon Ofor, Jessica Mason, Jordan R. Chen, Susan Srinivasan, Sundararajan Shivade, Chaitanya Moriarty, Jack Cohen, Joseph Paul |
| contents | Hallucinations in large language models (LLMs) during summarization of patient-clinician dialogues pose significant risks to patient care and clinical decision-making. However, the phenomenon remains understudied in the clinical domain, with uncertainty surrounding the applicability of general-domain hallucination detectors. The rarity and randomness of hallucinations further complicate their investigation. In this paper, we conduct an evaluation of hallucination detection methods in the medical domain, and construct two datasets for the purpose: A fact-controlled Leave-N-out dataset -- generated by systematically removing facts from source dialogues to induce hallucinated content in summaries; and a natural hallucination dataset -- arising organically during LLM-based medical summarization. We show that general-domain detectors struggle to detect clinical hallucinations, and that performance on fact-controlled hallucinations does not reliably predict effectiveness on natural hallucinations. We then develop fact-based approaches that count hallucinations, offering explainability not available with existing methods. Notably, our LLM-based detectors, which we developed using fact-controlled hallucinations, generalize well to detecting real-world clinical hallucinations. This research contributes a suite of specialized metrics supported by expert-annotated datasets to advance faithful clinical summarization systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_00448 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Fact-Controlled Diagnosis of Hallucinations in Medical Text Summarization BN, Suhas Shing, Han-Chin Xu, Lei Strong, Mitch Burnsky, Jon Ofor, Jessica Mason, Jordan R. Chen, Susan Srinivasan, Sundararajan Shivade, Chaitanya Moriarty, Jack Cohen, Joseph Paul Computation and Language Hallucinations in large language models (LLMs) during summarization of patient-clinician dialogues pose significant risks to patient care and clinical decision-making. However, the phenomenon remains understudied in the clinical domain, with uncertainty surrounding the applicability of general-domain hallucination detectors. The rarity and randomness of hallucinations further complicate their investigation. In this paper, we conduct an evaluation of hallucination detection methods in the medical domain, and construct two datasets for the purpose: A fact-controlled Leave-N-out dataset -- generated by systematically removing facts from source dialogues to induce hallucinated content in summaries; and a natural hallucination dataset -- arising organically during LLM-based medical summarization. We show that general-domain detectors struggle to detect clinical hallucinations, and that performance on fact-controlled hallucinations does not reliably predict effectiveness on natural hallucinations. We then develop fact-based approaches that count hallucinations, offering explainability not available with existing methods. Notably, our LLM-based detectors, which we developed using fact-controlled hallucinations, generalize well to detecting real-world clinical hallucinations. This research contributes a suite of specialized metrics supported by expert-annotated datasets to advance faithful clinical summarization systems. |
| title | Fact-Controlled Diagnosis of Hallucinations in Medical Text Summarization |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2506.00448 |