Medical Hallucinations in Foundation Models and Their Impact on Healthcare
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914128788979712 |
|---|---|
| author | Kim, Yubin Jeong, Hyewon Chen, Shan Li, Shuyue Stella Park, Chanwoo Lu, Mingyu Alhamoud, Kumail Mun, Jimin Grau, Cristina Jung, Minseok Gameiro, Rodrigo Fan, Lizhou Park, Eugene Lin, Tristan Yoon, Joonsik Yoon, Wonjin Sap, Maarten Tsvetkov, Yulia Liang, Paul Xu, Xuhai Liu, Xin Park, Chunjong Lee, Hyeonhoon Park, Hae Won McDuff, Daniel Tulebaev, Samir Breazeal, Cynthia |
| author_facet | Kim, Yubin Jeong, Hyewon Chen, Shan Li, Shuyue Stella Park, Chanwoo Lu, Mingyu Alhamoud, Kumail Mun, Jimin Grau, Cristina Jung, Minseok Gameiro, Rodrigo Fan, Lizhou Park, Eugene Lin, Tristan Yoon, Joonsik Yoon, Wonjin Sap, Maarten Tsvetkov, Yulia Liang, Paul Xu, Xuhai Liu, Xin Park, Chunjong Lee, Hyeonhoon Park, Hae Won McDuff, Daniel Tulebaev, Samir Breazeal, Cynthia |
| contents | Hallucinations in foundation models arise from autoregressive training objectives that prioritize token-likelihood optimization over epistemic accuracy, fostering overconfidence and poorly calibrated uncertainty. We define medical hallucination as any model-generated output that is factually incorrect, logically inconsistent, or unsupported by authoritative clinical evidence in ways that could alter clinical decisions. We evaluated 11 foundation models (7 general-purpose, 4 medical-specialized) across seven medical hallucination tasks spanning medical reasoning and biomedical information retrieval. General-purpose models achieved significantly higher proportions of hallucination-free responses than medical-specialized models (median: 76.6% vs 51.3%, difference = 25.2%, 95% CI: 18.7-31.3%, Mann-Whitney U = 27.0, p = 0.012, rank-biserial r = -0.64). Top-performing models such as Gemini-2.5 Pro exceeded 97% accuracy when augmented with chain-of-thought prompting (base: 87.6%), while medical-specialized models like MedGemma ranged from 28.6-61.9% despite explicit training on medical corpora. Chain-of-thought reasoning significantly reduced hallucinations in 86.4% of tested comparisons after FDR correction (q < 0.05), demonstrating that explicit reasoning traces enable self-verification and error detection. Physician audits confirmed that 64-72% of residual hallucinations stemmed from causal or temporal reasoning failures rather than knowledge gaps. A global survey of clinicians (n = 70) validated real-world impact: 91.8% had encountered medical hallucinations, and 84.7% considered them capable of causing patient harm. The underperformance of medical-specialized models despite domain training indicates that safety emerges from sophisticated reasoning capabilities and broad knowledge integration developed during large-scale pre-training, not from narrow optimization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_05777 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Medical Hallucinations in Foundation Models and Their Impact on Healthcare Kim, Yubin Jeong, Hyewon Chen, Shan Li, Shuyue Stella Park, Chanwoo Lu, Mingyu Alhamoud, Kumail Mun, Jimin Grau, Cristina Jung, Minseok Gameiro, Rodrigo Fan, Lizhou Park, Eugene Lin, Tristan Yoon, Joonsik Yoon, Wonjin Sap, Maarten Tsvetkov, Yulia Liang, Paul Xu, Xuhai Liu, Xin Park, Chunjong Lee, Hyeonhoon Park, Hae Won McDuff, Daniel Tulebaev, Samir Breazeal, Cynthia Computation and Language Artificial Intelligence Computers and Society Hallucinations in foundation models arise from autoregressive training objectives that prioritize token-likelihood optimization over epistemic accuracy, fostering overconfidence and poorly calibrated uncertainty. We define medical hallucination as any model-generated output that is factually incorrect, logically inconsistent, or unsupported by authoritative clinical evidence in ways that could alter clinical decisions. We evaluated 11 foundation models (7 general-purpose, 4 medical-specialized) across seven medical hallucination tasks spanning medical reasoning and biomedical information retrieval. General-purpose models achieved significantly higher proportions of hallucination-free responses than medical-specialized models (median: 76.6% vs 51.3%, difference = 25.2%, 95% CI: 18.7-31.3%, Mann-Whitney U = 27.0, p = 0.012, rank-biserial r = -0.64). Top-performing models such as Gemini-2.5 Pro exceeded 97% accuracy when augmented with chain-of-thought prompting (base: 87.6%), while medical-specialized models like MedGemma ranged from 28.6-61.9% despite explicit training on medical corpora. Chain-of-thought reasoning significantly reduced hallucinations in 86.4% of tested comparisons after FDR correction (q < 0.05), demonstrating that explicit reasoning traces enable self-verification and error detection. Physician audits confirmed that 64-72% of residual hallucinations stemmed from causal or temporal reasoning failures rather than knowledge gaps. A global survey of clinicians (n = 70) validated real-world impact: 91.8% had encountered medical hallucinations, and 84.7% considered them capable of causing patient harm. The underperformance of medical-specialized models despite domain training indicates that safety emerges from sophisticated reasoning capabilities and broad knowledge integration developed during large-scale pre-training, not from narrow optimization. |
| title | Medical Hallucinations in Foundation Models and Their Impact on Healthcare |
| topic | Computation and Language Artificial Intelligence Computers and Society |
| url | https://arxiv.org/abs/2503.05777 |