Is It Bad to Work All the Time? Cross-Cultural Evaluation of Social Norm Biases in GPT-4
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915303180468224 |
|---|---|
| author | Liu, Zhuozhuo Joy Samir, Farhan Bhatia, Mehar Nelson, Laura K. Shwartz, Vered |
| author_facet | Liu, Zhuozhuo Joy Samir, Farhan Bhatia, Mehar Nelson, Laura K. Shwartz, Vered |
| contents | LLMs have been demonstrated to align with the values of Western or North American cultures. Prior work predominantly showed this effect through leveraging surveys that directly ask (originally people and now also LLMs) about their values. However, it is hard to believe that LLMs would consistently apply those values in real-world scenarios. To address that, we take a bottom-up approach, asking LLMs to reason about cultural norms in narratives from different cultures. We find that GPT-4 tends to generate norms that, while not necessarily incorrect, are significantly less culture-specific. In addition, while it avoids overtly generating stereotypes, the stereotypical representations of certain cultures are merely hidden rather than suppressed in the model, and such stereotypes can be easily recovered. Addressing these challenges is a crucial step towards developing LLMs that fairly serve their diverse user base. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_18322 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Is It Bad to Work All the Time? Cross-Cultural Evaluation of Social Norm Biases in GPT-4 Liu, Zhuozhuo Joy Samir, Farhan Bhatia, Mehar Nelson, Laura K. Shwartz, Vered Computation and Language Artificial Intelligence LLMs have been demonstrated to align with the values of Western or North American cultures. Prior work predominantly showed this effect through leveraging surveys that directly ask (originally people and now also LLMs) about their values. However, it is hard to believe that LLMs would consistently apply those values in real-world scenarios. To address that, we take a bottom-up approach, asking LLMs to reason about cultural norms in narratives from different cultures. We find that GPT-4 tends to generate norms that, while not necessarily incorrect, are significantly less culture-specific. In addition, while it avoids overtly generating stereotypes, the stereotypical representations of certain cultures are merely hidden rather than suppressed in the model, and such stereotypes can be easily recovered. Addressing these challenges is a crucial step towards developing LLMs that fairly serve their diverse user base. |
| title | Is It Bad to Work All the Time? Cross-Cultural Evaluation of Social Norm Biases in GPT-4 |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2505.18322 |