Evaluating Biases in Context-Dependent Health Questions
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911791488958464 |
|---|---|
| author | Levy, Sharon Karver, Tahilin Sanchez Adler, William D. Kaufman, Michelle R. Dredze, Mark |
| author_facet | Levy, Sharon Karver, Tahilin Sanchez Adler, William D. Kaufman, Michelle R. Dredze, Mark |
| contents | Chat-based large language models have the opportunity to empower individuals lacking high-quality healthcare access to receive personalized information across a variety of topics. However, users may ask underspecified questions that require additional context for a model to correctly answer. We study how large language model biases are exhibited through these contextual questions in the healthcare domain. To accomplish this, we curate a dataset of sexual and reproductive healthcare questions that are dependent on age, sex, and location attributes. We compare models' outputs with and without demographic context to determine group alignment among our contextual questions. Our experiments reveal biases in each of these attributes, where young adult female users are favored. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_04858 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Evaluating Biases in Context-Dependent Health Questions Levy, Sharon Karver, Tahilin Sanchez Adler, William D. Kaufman, Michelle R. Dredze, Mark Computation and Language Chat-based large language models have the opportunity to empower individuals lacking high-quality healthcare access to receive personalized information across a variety of topics. However, users may ask underspecified questions that require additional context for a model to correctly answer. We study how large language model biases are exhibited through these contextual questions in the healthcare domain. To accomplish this, we curate a dataset of sexual and reproductive healthcare questions that are dependent on age, sex, and location attributes. We compare models' outputs with and without demographic context to determine group alignment among our contextual questions. Our experiments reveal biases in each of these attributes, where young adult female users are favored. |
| title | Evaluating Biases in Context-Dependent Health Questions |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2403.04858 |