Image de couverture de livre

Culture is Everywhere: A Call for Intentionally Cultural Evaluation

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Oh, Juhyun, Cha, Inha, Saxon, Michael, Lim, Hyunseung, Bhatt, Shaily, Oh, Alice
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

The Generative AI Paradox on Evaluation: What It Can Solve, It May Not Evaluate
par: Oh, Juhyun, et autres
Publié: (2024)

Extrinsic Evaluation of Cultural Competence in Large Language Models
par: Bhatt, Shaily, et autres
Publié: (2024)

Uncovering Factor Level Preferences to Improve Human-Model Alignment
par: Oh, Juhyun, et autres
Publié: (2024)

Flex-TravelPlanner: A Benchmark for Flexible Planning with Language Agents
par: Oh, Juhyun, et autres
Publié: (2025)

JuICE: A Benchmark for Evaluating LLM-Judge in Identifying Cultural Errors
par: Jin, Jiho, et autres
Publié: (2026)

Research Borderlands: Analysing Writing Across Research Cultures
par: Bhatt, Shaily, et autres
Publié: (2025)

Multi-FAct: Assessing Factuality of Multilingual LLMs using FActScore
par: Shafayat, Sheikh, et autres
Publié: (2024)

OLA: Output Language Alignment in Code-Switched LLM Interactions
par: Oh, Juhyun, et autres
Publié: (2026)

Spotting Out-of-Character Behavior: Atomic-Level Evaluation of Persona Fidelity in Open-Ended Generation
par: Shin, Jisu, et autres
Publié: (2025)

RoleConflictBench: A Benchmark of Role Conflict Scenarios for Evaluating LLMs' Contextual Sensitivity
par: Shin, Jisu, et autres
Publié: (2025)

CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Korean
par: Kim, Eunsu, et autres
Publié: (2024)

CAIRe: Cultural Attribution of Images by Retrieval-Augmented Evaluation
par: Yayavaram, Arnav, et autres
Publié: (2025)

FINEST: Improving LLM Responses to Sensitive Topics Through Fine-Grained Evaluation
par: Oh, Juhyun, et autres
Publié: (2026)

Are they lovers or friends? Evaluating LLMs' Social Reasoning in English and Korean Dialogues
par: Kim, Eunsu, et autres
Publié: (2025)

TALES: A Taxonomy and Analysis of Cultural Representations in LLM-generated Stories
par: Bhagat, Kirti, et autres
Publié: (2025)

When Tom Eats Kimchi: Evaluating Cultural Bias of Multimodal Large Language Models in Cultural Mixture Contexts
par: Kim, Jun Seong, et autres
Publié: (2025)

Can LLM Generate Culturally Relevant Commonsense QA Data? Case Study in Indonesian and Sundanese
par: Putri, Rifki Afina, et autres
Publié: (2024)

Benchmarks as Microscopes: A Call for Model Metrology
par: Saxon, Michael, et autres
Publié: (2024)

LLM-C3MOD: A Human-LLM Collaborative System for Cross-Cultural Hate Speech Moderation
par: Park, Junyeong, et autres
Publié: (2025)

Do You Know About My Nation? Investigating Multilingual Language Models' Cultural Literacy Through Factual Knowledge
par: Tanwar, Eshaan, et autres
Publié: (2025)

BLUCK: A Benchmark Dataset for Bengali Linguistic Understanding and Cultural Knowledge
par: Kabir, Daeen, et autres
Publié: (2025)

LLM-as-a-tutor in EFL Writing Education: Focusing on Evaluation of Student-LLM Interaction
par: Han, Jieun, et autres
Publié: (2023)

Entangled in Representations: Mechanistic Investigation of Cultural Biases in Large Language Models
par: Yu, Haeun, et autres
Publié: (2025)

Exploring Cross-Cultural Differences in English Hate Speech Annotations: From Dataset Construction to Analysis
par: Lee, Nayeon, et autres
Publié: (2023)

Survey of Cultural Awareness in Language Models: Text and Beyond
par: Pawar, Siddhesh, et autres
Publié: (2024)

Exploring Persona-dependent LLM Alignment for the Moral Machine Experiment
par: Kim, Jiseon, et autres
Publié: (2025)

Cultural Compass: A Framework for Organizing Societal Norms to Detect Violations in Human-AI Conversations
par: Cheng, Myra, et autres
Publié: (2026)

Social Bias Benchmark for Generation: A Comparison of Generation and QA-Based Evaluations
par: Jin, Jiho, et autres
Publié: (2025)

Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation
par: Singh, Shivalika, et autres
Publié: (2024)

Who Speaks Matters: Analysing the Influence of the Speaker's Ethnicity on Hate Classification
par: Malik, Ananya, et autres
Publié: (2024)

PapersPlease: A Benchmark for Evaluating Motivational Values of Large Language Models Based on ERG Theory
par: Myung, Junho, et autres
Publié: (2025)

A 2-step Framework for Automated Literary Translation Evaluation: Its Promises and Pitfalls
par: Shafayat, Sheikh, et autres
Publié: (2024)

CultureForest: Understanding and Evaluating Cultural Norm Grounded Reasoning in LLMs
par: Ye, Yangfan, et autres
Publié: (2026)

CURE: Cultural Understanding and Reasoning Evaluation - A Framework for "Thick" Culture Alignment Evaluation in LLMs
par: Vo, Truong, et autres
Publié: (2025)

On the Effect of Uncertainty on Layer-wise Inference Dynamics
par: Kim, Sunwoo, et autres
Publié: (2025)

MentalBench: A DSM-Grounded Benchmark for Evaluating Psychiatric Diagnostic Capability of Large Language Models
par: Song, Hoyun, et autres
Publié: (2026)

XL-SafetyBench: A Country-Grounded Cross-Cultural Benchmark for LLM Safety and Cultural Sensitivity
par: Choi, Dasol, et autres
Publié: (2026)

Can LLMs Express Personality Across Cultures? Introducing CulturalPersonas for Evaluating Trait Alignment
par: Dey, Priyanka, et autres
Publié: (2025)

KULTURE Bench: A Benchmark for Assessing Language Model in Korean Cultural Context
par: Wang, Xiaonan, et autres
Publié: (2024)

BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages
par: Myung, Junho, et autres
Publié: (2024)