Investigating Counterfactual Unfairness in LLMs towards Identities through Humor
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911609001082880 |
|---|---|
| author | Kim, Shubin Son, Yejin Park, Junyeong Ka, Keummin Lee, Seungbeen Lee, Jaeyoung Jang, Hyeju Oh, Alice Yu, Youngjae |
| author_facet | Kim, Shubin Son, Yejin Park, Junyeong Ka, Keummin Lee, Seungbeen Lee, Jaeyoung Jang, Hyeju Oh, Alice Yu, Youngjae |
| contents | Humor holds up a mirror to social perception: what we find funny often reflects who we are and how we judge others. When language models engage with humor, their reactions expose the social assumptions they have internalized from training data. In this paper, we investigate counterfactual unfairness through humor by observing how the model's responses change when we swap who speaks and who is addressed while holding other factors constant. Our framework spans three tasks: humor generation refusal, speaker intention inference, and relational/societal impact prediction, covering both identity-agnostic humor and identity-specific disparagement humor. We introduce interpretable bias metrics that capture asymmetric patterns under identity swaps. Experiments across state-of-the-art models reveal consistent relational disparities: jokes told by privileged speakers are refused up to 67.5% more often, judged as malicious 64.7% more frequently, and rated up to 1.5 points higher in social harm on a 5-point scale. These patterns highlight how sensitivity and stereotyping coexist in generative models, complicating efforts toward fairness and cultural alignment. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_18729 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Investigating Counterfactual Unfairness in LLMs towards Identities through Humor Kim, Shubin Son, Yejin Park, Junyeong Ka, Keummin Lee, Seungbeen Lee, Jaeyoung Jang, Hyeju Oh, Alice Yu, Youngjae Computation and Language Humor holds up a mirror to social perception: what we find funny often reflects who we are and how we judge others. When language models engage with humor, their reactions expose the social assumptions they have internalized from training data. In this paper, we investigate counterfactual unfairness through humor by observing how the model's responses change when we swap who speaks and who is addressed while holding other factors constant. Our framework spans three tasks: humor generation refusal, speaker intention inference, and relational/societal impact prediction, covering both identity-agnostic humor and identity-specific disparagement humor. We introduce interpretable bias metrics that capture asymmetric patterns under identity swaps. Experiments across state-of-the-art models reveal consistent relational disparities: jokes told by privileged speakers are refused up to 67.5% more often, judged as malicious 64.7% more frequently, and rated up to 1.5 points higher in social harm on a 5-point scale. These patterns highlight how sensitivity and stereotyping coexist in generative models, complicating efforts toward fairness and cultural alignment. |
| title | Investigating Counterfactual Unfairness in LLMs towards Identities through Humor |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2604.18729 |