A Toolbox for Surfacing Health Equity Harms and Biases in Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929528960450560 |
|---|---|
| author | Pfohl, Stephen R. Cole-Lewis, Heather Sayres, Rory Neal, Darlene Asiedu, Mercy Dieng, Awa Tomasev, Nenad Rashid, Qazi Mamunur Azizi, Shekoofeh Rostamzadeh, Negar McCoy, Liam G. Celi, Leo Anthony Liu, Yun Schaekermann, Mike Walton, Alanna Parrish, Alicia Nagpal, Chirag Singh, Preeti Dewitt, Akeiylah Mansfield, Philip Prakash, Sushant Heller, Katherine Karthikesalingam, Alan Semturs, Christopher Barral, Joelle Corrado, Greg Matias, Yossi Smith-Loud, Jamila Horn, Ivor Singhal, Karan |
| author_facet | Pfohl, Stephen R. Cole-Lewis, Heather Sayres, Rory Neal, Darlene Asiedu, Mercy Dieng, Awa Tomasev, Nenad Rashid, Qazi Mamunur Azizi, Shekoofeh Rostamzadeh, Negar McCoy, Liam G. Celi, Leo Anthony Liu, Yun Schaekermann, Mike Walton, Alanna Parrish, Alicia Nagpal, Chirag Singh, Preeti Dewitt, Akeiylah Mansfield, Philip Prakash, Sushant Heller, Katherine Karthikesalingam, Alan Semturs, Christopher Barral, Joelle Corrado, Greg Matias, Yossi Smith-Loud, Jamila Horn, Ivor Singhal, Karan |
| contents | Large language models (LLMs) hold promise to serve complex health information needs but also have the potential to introduce harm and exacerbate health disparities. Reliably evaluating equity-related model failures is a critical step toward developing systems that promote health equity. We present resources and methodologies for surfacing biases with potential to precipitate equity-related harms in long-form, LLM-generated answers to medical questions and conduct a large-scale empirical case study with the Med-PaLM 2 LLM. Our contributions include a multifactorial framework for human assessment of LLM-generated answers for biases, and EquityMedQA, a collection of seven datasets enriched for adversarial queries. Both our human assessment framework and dataset design process are grounded in an iterative participatory approach and review of Med-PaLM 2 answers. Through our empirical study, we find that our approach surfaces biases that may be missed via narrower evaluation approaches. Our experience underscores the importance of using diverse assessment methodologies and involving raters of varying backgrounds and expertise. While our approach is not sufficient to holistically assess whether the deployment of an AI system promotes equitable health outcomes, we hope that it can be leveraged and built upon towards a shared goal of LLMs that promote accessible and equitable healthcare. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_12025 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Toolbox for Surfacing Health Equity Harms and Biases in Large Language Models Pfohl, Stephen R. Cole-Lewis, Heather Sayres, Rory Neal, Darlene Asiedu, Mercy Dieng, Awa Tomasev, Nenad Rashid, Qazi Mamunur Azizi, Shekoofeh Rostamzadeh, Negar McCoy, Liam G. Celi, Leo Anthony Liu, Yun Schaekermann, Mike Walton, Alanna Parrish, Alicia Nagpal, Chirag Singh, Preeti Dewitt, Akeiylah Mansfield, Philip Prakash, Sushant Heller, Katherine Karthikesalingam, Alan Semturs, Christopher Barral, Joelle Corrado, Greg Matias, Yossi Smith-Loud, Jamila Horn, Ivor Singhal, Karan Computers and Society Computation and Language Machine Learning Large language models (LLMs) hold promise to serve complex health information needs but also have the potential to introduce harm and exacerbate health disparities. Reliably evaluating equity-related model failures is a critical step toward developing systems that promote health equity. We present resources and methodologies for surfacing biases with potential to precipitate equity-related harms in long-form, LLM-generated answers to medical questions and conduct a large-scale empirical case study with the Med-PaLM 2 LLM. Our contributions include a multifactorial framework for human assessment of LLM-generated answers for biases, and EquityMedQA, a collection of seven datasets enriched for adversarial queries. Both our human assessment framework and dataset design process are grounded in an iterative participatory approach and review of Med-PaLM 2 answers. Through our empirical study, we find that our approach surfaces biases that may be missed via narrower evaluation approaches. Our experience underscores the importance of using diverse assessment methodologies and involving raters of varying backgrounds and expertise. While our approach is not sufficient to holistically assess whether the deployment of an AI system promotes equitable health outcomes, we hope that it can be leveraged and built upon towards a shared goal of LLMs that promote accessible and equitable healthcare. |
| title | A Toolbox for Surfacing Health Equity Harms and Biases in Large Language Models |
| topic | Computers and Society Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2403.12025 |