Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.05613 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912404468662272 |
|---|---|
| author | Kantharuban, Anjali Milbauer, Jeremiah Sap, Maarten Strubell, Emma Neubig, Graham |
| author_facet | Kantharuban, Anjali Milbauer, Jeremiah Sap, Maarten Strubell, Emma Neubig, Graham |
| contents | While personalized recommendations are often desired by users, it can be difficult in practice to distinguish cases of bias from cases of personalization: we find that models generate racially stereotypical recommendations regardless of whether the user revealed their identity intentionally through explicit indications or unintentionally through implicit cues. We demonstrate that when people use large language models (LLMs) to generate recommendations, the LLMs produce responses that reflect both what the user wants and who the user is. We argue that chatbots ought to transparently indicate when recommendations are influenced by a user's revealed identity characteristics, but observe that they currently fail to do so. Our experiments show that even though a user's revealed identity significantly influences model recommendations (p < 0.001), model responses obfuscate this fact in response to user queries. This bias and lack of transparency occurs consistently across multiple popular consumer LLMs and for four American racial groups. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_05613 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Stereotype or Personalization? User Identity Biases Chatbot Recommendations Kantharuban, Anjali Milbauer, Jeremiah Sap, Maarten Strubell, Emma Neubig, Graham Computation and Language While personalized recommendations are often desired by users, it can be difficult in practice to distinguish cases of bias from cases of personalization: we find that models generate racially stereotypical recommendations regardless of whether the user revealed their identity intentionally through explicit indications or unintentionally through implicit cues. We demonstrate that when people use large language models (LLMs) to generate recommendations, the LLMs produce responses that reflect both what the user wants and who the user is. We argue that chatbots ought to transparently indicate when recommendations are influenced by a user's revealed identity characteristics, but observe that they currently fail to do so. Our experiments show that even though a user's revealed identity significantly influences model recommendations (p < 0.001), model responses obfuscate this fact in response to user queries. This bias and lack of transparency occurs consistently across multiple popular consumer LLMs and for four American racial groups. |
| title | Stereotype or Personalization? User Identity Biases Chatbot Recommendations |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2410.05613 |