Are they lovers or friends? Evaluating LLMs' Social Reasoning in English and Korean Dialogues

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kim, Eunsu, Park, Junyeong, Oh, Juhyun, Park, Kiwoong, Song, Seyoung, Doğruöz, A. Seza, Oh, Alice, Kim, Najoung
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Flex-TravelPlanner: A Benchmark for Flexible Planning with Language Agents
by: Oh, Juhyun, et al.
Published: (2025)

LoCar: Localization-Aware Evaluation of In-Vehicle Assistants through Fine-Grained Sociolinguistic Control
by: Jeong, Seogyeong, et al.
Published: (2026)

Multi-FAct: Assessing Factuality of Multilingual LLMs using FActScore
by: Shafayat, Sheikh, et al.
Published: (2024)

The Generative AI Paradox on Evaluation: What It Can Solve, It May Not Evaluate
by: Oh, Juhyun, et al.
Published: (2024)

Spotting Out-of-Character Behavior: Atomic-Level Evaluation of Persona Fidelity in Open-Ended Generation
by: Shin, Jisu, et al.
Published: (2025)

Open Korean Historical Corpus: A Millennia-Scale Diachronic Collection of Public Domain Texts
by: Song, Seyoung, et al.
Published: (2025)

RoleConflictBench: A Benchmark of Role Conflict Scenarios for Evaluating LLMs' Contextual Sensitivity
by: Shin, Jisu, et al.
Published: (2025)

LLM-C3MOD: A Human-LLM Collaborative System for Cross-Cultural Hate Speech Moderation
by: Park, Junyeong, et al.
Published: (2025)

CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Korean
by: Kim, Eunsu, et al.
Published: (2024)

MUG-Eval: A Proxy Evaluation Framework for Multilingual Generation Capabilities in Any Language
by: Song, Seyoung, et al.
Published: (2025)

Uncovering Factor Level Preferences to Improve Human-Model Alignment
by: Oh, Juhyun, et al.
Published: (2024)

A Typology of Synthetic Datasets for Dialogue Processing in Clinical Contexts
by: Bedrick, Steven, et al.
Published: (2025)

Single- vs. Dual-Prompt Dialogue Generation with LLMs for Job Interviews in Human Resources
by: De Baer, Joachim, et al.
Published: (2025)

JuICE: A Benchmark for Evaluating LLM-Judge in Identifying Cultural Errors
by: Jin, Jiho, et al.
Published: (2026)

When Tom Eats Kimchi: Evaluating Cultural Bias of Multimodal Large Language Models in Cultural Mixture Contexts
by: Kim, Jun Seong, et al.
Published: (2025)

Leveraging LLMs for Translating and Classifying Mental Health Data
by: Skianis, Konstantinos, et al.
Published: (2024)

Readability Measures and Automatic Text Simplification: In the Search of a Construct
by: Cardon, Rémi, et al.
Published: (2025)

HERITAGE: An End-to-End Web Platform for Processing Korean Historical Documents in Hanja
by: Song, Seyoung, et al.
Published: (2025)

Building Multilingual Datasets for Predicting Mental Health Severity through LLMs: Prospects and Challenges
by: Skianis, Konstantinos, et al.
Published: (2024)

GECKO: Generative Language Model for English, Code and Korean
by: Oh, Sungwoo, et al.
Published: (2024)

LLM-as-an-Interviewer: Beyond Static Testing Through Dynamic LLM Evaluation
by: Kim, Eunsu, et al.
Published: (2024)

Does Generative AI speak Nigerian-Pidgin?: Issues about Representativeness and Bias for Multilingualism in LLMs
by: Adelani, David Ifeoluwa, et al.
Published: (2024)

OLA: Output Language Alignment in Code-Switched LLM Interactions
by: Oh, Juhyun, et al.
Published: (2026)

Investigating Counterfactual Unfairness in LLMs towards Identities through Humor
by: Kim, Shubin, et al.
Published: (2026)

Culture is Everywhere: A Call for Intentionally Cultural Evaluation
by: Oh, Juhyun, et al.
Published: (2025)

Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs' Legal Reasoning Capabilities
by: Oh, Hongseok, et al.
Published: (2025)

FINEST: Improving LLM Responses to Sensitive Topics Through Fine-Grained Evaluation
by: Oh, Juhyun, et al.
Published: (2026)

BenchHub: A Unified Benchmark Suite for Holistic and Customizable LLM Evaluation
by: Kim, Eunsu, et al.
Published: (2025)

Can Code-Switched Texts Activate a Knowledge Switch in LLMs? A Case Study on English-Korean Code-Switching
by: Kim, Seoyeon, et al.
Published: (2024)

Open Ko-LLM Leaderboard2: Bridging Foundational and Practical Evaluation for Korean LLMs
by: Kim, Hyeonwoo, et al.
Published: (2024)

Thunder-LLM: Efficiently Adapting LLMs to Korean with Minimal Resources
by: Kim, Jinpyo, et al.
Published: (2025)

Ko-MuSR: A Multistep Soft Reasoning Benchmark for LLMs Capable of Understanding Korean
by: Park, Chanwoo, et al.
Published: (2025)

Trans-EnV: A Framework for Evaluating the Linguistic Robustness of LLMs Against English Varieties
by: Lee, Jiyoung, et al.
Published: (2025)

KORMo: Korean Open Reasoning Model for Everyone
by: Kim, Minjun, et al.
Published: (2025)

Shared Heritage, Distinct Writing: Rethinking Resource Selection for East Asian Historical Documents
by: Song, Seyoung, et al.
Published: (2024)

Evaluating Multimodal Generative AI with Korean Educational Standards
by: Park, Sanghee, et al.
Published: (2025)

PapersPlease: A Benchmark for Evaluating Motivational Values of Large Language Models Based on ERG Theory
by: Myung, Junho, et al.
Published: (2025)

Rethinking Test-Time Scaling for Medical AI: Model and Task-Aware Strategies for LLMs and VLMs
by: Oh, Gyutaek, et al.
Published: (2025)

One-Topic-Doesn't-Fit-All: Transcreating Reading Comprehension Test for Personalized Learning
by: Han, Jieun, et al.
Published: (2025)

Is artificial intelligence still intelligence? LLMs generalize to novel adjective-noun pairs, but don't mimic the full human distribution
by: Ross, Hayley, et al.
Published: (2024)