Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2604.00014 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911558528925696 |
|---|---|
| author | Ni, Congning Qadir, Sarvech Steitz, Bryan Vaidya, Mihir Sachin Song, Qingyuan Xia, Lantian Mulvaney, Shelagh Liu, Siru Ryu, Hyeyoung Hecht, Leah Bucher, Amy Symons, Christopher Novak, Laurie Rose, Susannah L. Kantarcioglu, Murat Malin, Bradley Yin, Zhijun |
| author_facet | Ni, Congning Qadir, Sarvech Steitz, Bryan Vaidya, Mihir Sachin Song, Qingyuan Xia, Lantian Mulvaney, Shelagh Liu, Siru Ryu, Hyeyoung Hecht, Leah Bucher, Amy Symons, Christopher Novak, Laurie Rose, Susannah L. Kantarcioglu, Murat Malin, Bradley Yin, Zhijun |
| contents | Mental health concerns are often expressed outside clinical settings, including in high-distress help seeking, where safety-critical guidance may be needed. Consumer health informatics systems increasingly incorporate large language models (LLMs) for mental health question answering, yet many evaluations underrepresent narrative, high-distress inquiries. We introduce UTCO (User, Topic, Context, Tone), a prompt construction framework that represents an inquiry as four controllable elements for systematic stress testing. Using 2,075 UTCO-generated prompts, we evaluated Llama 3.3 and annotated hallucinations (fabricated or incorrect clinical content) and omissions (missing clinically necessary or safety-critical guidance). Hallucinations occurred in 6.5% of responses and omissions in 13.2%, with omissions concentrated in crisis and suicidal ideation prompts. Across regression, element-specific matching, and similarity-matched comparisons, failures were most consistently associated with context and tone, while user-background indicators showed no systematic differences after balancing. These findings support evaluating omissions as a primary safety outcome and moving beyond static benchmark question sets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_00014 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Disentangling Prompt Element Level Risk Factors for Hallucinations and Omissions in Mental Health LLM Responses Ni, Congning Qadir, Sarvech Steitz, Bryan Vaidya, Mihir Sachin Song, Qingyuan Xia, Lantian Mulvaney, Shelagh Liu, Siru Ryu, Hyeyoung Hecht, Leah Bucher, Amy Symons, Christopher Novak, Laurie Rose, Susannah L. Kantarcioglu, Murat Malin, Bradley Yin, Zhijun Computation and Language Human-Computer Interaction Mental health concerns are often expressed outside clinical settings, including in high-distress help seeking, where safety-critical guidance may be needed. Consumer health informatics systems increasingly incorporate large language models (LLMs) for mental health question answering, yet many evaluations underrepresent narrative, high-distress inquiries. We introduce UTCO (User, Topic, Context, Tone), a prompt construction framework that represents an inquiry as four controllable elements for systematic stress testing. Using 2,075 UTCO-generated prompts, we evaluated Llama 3.3 and annotated hallucinations (fabricated or incorrect clinical content) and omissions (missing clinically necessary or safety-critical guidance). Hallucinations occurred in 6.5% of responses and omissions in 13.2%, with omissions concentrated in crisis and suicidal ideation prompts. Across regression, element-specific matching, and similarity-matched comparisons, failures were most consistently associated with context and tone, while user-background indicators showed no systematic differences after balancing. These findings support evaluating omissions as a primary safety outcome and moving beyond static benchmark question sets. |
| title | Disentangling Prompt Element Level Risk Factors for Hallucinations and Omissions in Mental Health LLM Responses |
| topic | Computation and Language Human-Computer Interaction |
| url | https://arxiv.org/abs/2604.00014 |