Cultural Compass: A Framework for Organizing Societal Norms to Detect Violations in Human-AI Conversations
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912819735166976 |
|---|---|
| author | Cheng, Myra Prabhakaran, Vinodkumar Oh, Alice Stepanyan, Hayk Verma, Aishwarya Kalia, Charu van Liemt, Erin MacMurray Dev, Sunipa |
| author_facet | Cheng, Myra Prabhakaran, Vinodkumar Oh, Alice Stepanyan, Hayk Verma, Aishwarya Kalia, Charu van Liemt, Erin MacMurray Dev, Sunipa |
| contents | Generative AI models ought to be useful and safe across cross-cultural contexts. One critical step toward this goal is understanding how AI models adhere to sociocultural norms. While this challenge has gained attention in NLP, existing work lacks both nuance and coverage in understanding and evaluating models' norm adherence. We address these gaps by introducing a taxonomy of norms that clarifies their contexts (e.g., distinguishing between human-human norms that models should recognize and human-AI interactional norms that apply to the human-AI interaction itself), specifications (e.g., relevant domains), and mechanisms (e.g., modes of enforcement). We demonstrate how our taxonomy can be operationalized to automatically evaluate models' norm adherence in naturalistic, open-ended settings. Our exploratory analyses suggest that state-of-the-art models frequently violate norms, though violation rates vary by model, interactional context, and country. We further show that violation rates also vary by prompt intent and situational framing. Our taxonomy and demonstrative evaluation pipeline enable nuanced, context-sensitive evaluation of cultural norm adherence in realistic settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_07973 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Cultural Compass: A Framework for Organizing Societal Norms to Detect Violations in Human-AI Conversations Cheng, Myra Prabhakaran, Vinodkumar Oh, Alice Stepanyan, Hayk Verma, Aishwarya Kalia, Charu van Liemt, Erin MacMurray Dev, Sunipa Computers and Society Artificial Intelligence Computation and Language Generative AI models ought to be useful and safe across cross-cultural contexts. One critical step toward this goal is understanding how AI models adhere to sociocultural norms. While this challenge has gained attention in NLP, existing work lacks both nuance and coverage in understanding and evaluating models' norm adherence. We address these gaps by introducing a taxonomy of norms that clarifies their contexts (e.g., distinguishing between human-human norms that models should recognize and human-AI interactional norms that apply to the human-AI interaction itself), specifications (e.g., relevant domains), and mechanisms (e.g., modes of enforcement). We demonstrate how our taxonomy can be operationalized to automatically evaluate models' norm adherence in naturalistic, open-ended settings. Our exploratory analyses suggest that state-of-the-art models frequently violate norms, though violation rates vary by model, interactional context, and country. We further show that violation rates also vary by prompt intent and situational framing. Our taxonomy and demonstrative evaluation pipeline enable nuanced, context-sensitive evaluation of cultural norm adherence in realistic settings. |
| title | Cultural Compass: A Framework for Organizing Societal Norms to Detect Violations in Human-AI Conversations |
| topic | Computers and Society Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2601.07973 |