Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.08073 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913687666688000 |
|---|---|
| author | Akhras, Naomi Antaki, Fares Mottet, Fannie Nguyen, Olivia Sawhney, Shyam Bajwah, Sabrina Davies, Joanna M |
| author_facet | Akhras, Naomi Antaki, Fares Mottet, Fannie Nguyen, Olivia Sawhney, Shyam Bajwah, Sabrina Davies, Joanna M |
| contents | Bias and inequity in palliative care disproportionately affect marginalised groups. Large language models (LLMs), such as GPT-4o, hold potential to enhance care but risk perpetuating biases present in their training data. This study aimed to systematically evaluate whether GPT-4o propagates biases in palliative care responses using adversarially designed datasets. In July 2024, GPT-4o was probed using the Palliative Care Adversarial Dataset (PCAD), and responses were evaluated by three palliative care experts in Canada and the United Kingdom using validated bias rubrics. The PCAD comprised PCAD-Direct (100 adversarial questions) and PCAD-Counterfactual (84 paired scenarios). These datasets targeted four care dimensions (access to care, pain management, advance care planning, and place of death preferences) and three identity axes (ethnicity, age, and diagnosis). Bias was detected in a substantial proportion of responses. For adversarial questions, the pooled bias rate was 0.33 (95% confidence interval [CI]: 0.28, 0.38); "allows biased premise" was the most frequently identified source of bias (0.47; 95% CI: 0.39, 0.55), such as failing to challenge stereotypes. For counterfactual scenarios, the pooled bias rate was 0.26 (95% CI: 0.20, 0.31), with "potential for withholding" as the most frequently identified source of bias (0.25; 95% CI: 0.18, 0.34), such as withholding interventions based on identity. Bias rates were consistent across care dimensions and identity axes. GPT-4o perpetuates biases in palliative care, with implications for clinical decision-making and equity. The PCAD datasets provide novel tools to assess and address LLM bias in palliative care. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_08073 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Large language models perpetuate bias in palliative care: development and analysis of the Palliative Care Adversarial Dataset (PCAD) Akhras, Naomi Antaki, Fares Mottet, Fannie Nguyen, Olivia Sawhney, Shyam Bajwah, Sabrina Davies, Joanna M Computers and Society Bias and inequity in palliative care disproportionately affect marginalised groups. Large language models (LLMs), such as GPT-4o, hold potential to enhance care but risk perpetuating biases present in their training data. This study aimed to systematically evaluate whether GPT-4o propagates biases in palliative care responses using adversarially designed datasets. In July 2024, GPT-4o was probed using the Palliative Care Adversarial Dataset (PCAD), and responses were evaluated by three palliative care experts in Canada and the United Kingdom using validated bias rubrics. The PCAD comprised PCAD-Direct (100 adversarial questions) and PCAD-Counterfactual (84 paired scenarios). These datasets targeted four care dimensions (access to care, pain management, advance care planning, and place of death preferences) and three identity axes (ethnicity, age, and diagnosis). Bias was detected in a substantial proportion of responses. For adversarial questions, the pooled bias rate was 0.33 (95% confidence interval [CI]: 0.28, 0.38); "allows biased premise" was the most frequently identified source of bias (0.47; 95% CI: 0.39, 0.55), such as failing to challenge stereotypes. For counterfactual scenarios, the pooled bias rate was 0.26 (95% CI: 0.20, 0.31), with "potential for withholding" as the most frequently identified source of bias (0.25; 95% CI: 0.18, 0.34), such as withholding interventions based on identity. Bias rates were consistent across care dimensions and identity axes. GPT-4o perpetuates biases in palliative care, with implications for clinical decision-making and equity. The PCAD datasets provide novel tools to assess and address LLM bias in palliative care. |
| title | Large language models perpetuate bias in palliative care: development and analysis of the Palliative Care Adversarial Dataset (PCAD) |
| topic | Computers and Society |
| url | https://arxiv.org/abs/2502.08073 |