Different Demographic Cues Yield Inconsistent Conclusions About LLM Personalization and Bias
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917356046909440 |
|---|---|
| author | Tonneau, Manuel Seghal, Neil K. R. Malhotra, Niyati Kazemi, Sharif Orozco-Olvera, Victor Boudet, Ana María Muñoz Subramanian, Lakshmi Fraiberger, Samuel P. Guntuku, Sharath Chandra Hofmann, Valentin |
| author_facet | Tonneau, Manuel Seghal, Neil K. R. Malhotra, Niyati Kazemi, Sharif Orozco-Olvera, Victor Boudet, Ana María Muñoz Subramanian, Lakshmi Fraiberger, Samuel P. Guntuku, Sharath Chandra Hofmann, Valentin |
| contents | Demographic cue-based evaluation is widely used to study how large language models (LLMs) adapt their responses to signaled demographic attributes within and across groups. This approach typically relies on a single cue (e.g., names) as a proxy for group membership, implicitly treating different cues as interchangeable operationalizations of the same identity-conditioned behavior. We test this assumption in realistic advice-seeking interactions spanning 14.8 million prompts, focusing on race and gender in a U.S. context. We find that cues for the same group induce only partially overlapping changes in model responses, yielding inconsistent conclusions about personalization, while bias conclusions are unstable, with both magnitude and direction of group differences varying across cues. We further show that these inconsistencies reflect differences in cue-group association strength and linguistic features bundled within cues that shape model responses. Together, our findings suggest that demographic conditioning in LLMs is not a cue-invariant category-level parameter but depends fundamentally on how identity is cued, reflecting responses to linguistic signals rather than stable demographic categories. We therefore advocate multi-cue, mechanism-aware evaluations for robust and interpretable claims about demographic variation in LLM responses. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_18486 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Different Demographic Cues Yield Inconsistent Conclusions About LLM Personalization and Bias Tonneau, Manuel Seghal, Neil K. R. Malhotra, Niyati Kazemi, Sharif Orozco-Olvera, Victor Boudet, Ana María Muñoz Subramanian, Lakshmi Fraiberger, Samuel P. Guntuku, Sharath Chandra Hofmann, Valentin Computation and Language Computers and Society Demographic cue-based evaluation is widely used to study how large language models (LLMs) adapt their responses to signaled demographic attributes within and across groups. This approach typically relies on a single cue (e.g., names) as a proxy for group membership, implicitly treating different cues as interchangeable operationalizations of the same identity-conditioned behavior. We test this assumption in realistic advice-seeking interactions spanning 14.8 million prompts, focusing on race and gender in a U.S. context. We find that cues for the same group induce only partially overlapping changes in model responses, yielding inconsistent conclusions about personalization, while bias conclusions are unstable, with both magnitude and direction of group differences varying across cues. We further show that these inconsistencies reflect differences in cue-group association strength and linguistic features bundled within cues that shape model responses. Together, our findings suggest that demographic conditioning in LLMs is not a cue-invariant category-level parameter but depends fundamentally on how identity is cued, reflecting responses to linguistic signals rather than stable demographic categories. We therefore advocate multi-cue, mechanism-aware evaluations for robust and interpretable claims about demographic variation in LLM responses. |
| title | Different Demographic Cues Yield Inconsistent Conclusions About LLM Personalization and Bias |
| topic | Computation and Language Computers and Society |
| url | https://arxiv.org/abs/2601.18486 |