The Biased Oracle: Assessing LLMs' Understandability and Empathy in Medical Diagnoses
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915592330543104 |
|---|---|
| author | Yao, Jianzhou Liu, Shunchang Drui, Guillaume Pettersson, Rikard Blasimme, Alessandro Kijewski, Sara |
| author_facet | Yao, Jianzhou Liu, Shunchang Drui, Guillaume Pettersson, Rikard Blasimme, Alessandro Kijewski, Sara |
| contents | Large language models (LLMs) show promise for supporting clinicians in diagnostic communication by generating explanations and guidance for patients. Yet their ability to produce outputs that are both understandable and empathetic remains uncertain. We evaluate two leading LLMs on medical diagnostic scenarios, assessing understandability using readability metrics as a proxy and empathy through LLM-as-a-Judge ratings compared to human evaluations. The results indicate that LLMs adapt explanations to socio-demographic variables and patient conditions. However, they also generate overly complex content and display biased affective empathy, leading to uneven accessibility and support. These patterns underscore the need for systematic calibration to ensure equitable patient communication. The code and data are released: https://github.com/Jeffateth/Biased_Oracle |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_00924 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Biased Oracle: Assessing LLMs' Understandability and Empathy in Medical Diagnoses Yao, Jianzhou Liu, Shunchang Drui, Guillaume Pettersson, Rikard Blasimme, Alessandro Kijewski, Sara Computation and Language Large language models (LLMs) show promise for supporting clinicians in diagnostic communication by generating explanations and guidance for patients. Yet their ability to produce outputs that are both understandable and empathetic remains uncertain. We evaluate two leading LLMs on medical diagnostic scenarios, assessing understandability using readability metrics as a proxy and empathy through LLM-as-a-Judge ratings compared to human evaluations. The results indicate that LLMs adapt explanations to socio-demographic variables and patient conditions. However, they also generate overly complex content and display biased affective empathy, leading to uneven accessibility and support. These patterns underscore the need for systematic calibration to ensure equitable patient communication. The code and data are released: https://github.com/Jeffateth/Biased_Oracle |
| title | The Biased Oracle: Assessing LLMs' Understandability and Empathy in Medical Diagnoses |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2511.00924 |