Cover Image

Rethinking Role-Playing Evaluation: Anonymous Benchmarking and a Systematic Study of Personality Effects

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Peng, Ji-Lun, Chen, Yun-Nung
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Exploring Personality-Aware Interactions in Salesperson Dialogue Agents
by: Cheng, Sijia, et al.
Published: (2025)

Injecting Salesperson's Dialogue Strategies in Large Language Models with Chain-of-Thought Reasoning
by: Chang, Wen-Yu, et al.
Published: (2024)

Balancing Knowledge Delivery and Emotional Comfort in Healthcare Conversational Systems
by: Tsai, Shang-Chi, et al.
Published: (2025)

FactAlign: Long-form Factuality Alignment of Large Language Models
by: Huang, Chao-Wei, et al.
Published: (2024)

RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models
by: Wang, Zekun Moore, et al.
Published: (2023)

From Persona to Personalization: A Survey on Role-Playing Language Agents
by: Chen, Jiangjie, et al.
Published: (2024)

VoxRole: A Comprehensive Benchmark for Evaluating Speech-Based Role-Playing Agents
by: Wu, Weihao, et al.
Published: (2025)

CharacterBox: Evaluating the Role-Playing Capabilities of LLMs in Text-Based Virtual Worlds
by: Wang, Lei, et al.
Published: (2024)

Role-Playing Evaluation for Large Language Models
by: Boudouri, Yassine El, et al.
Published: (2025)

I Need Help! Evaluating LLM's Ability to Ask for Users' Support: A Case Study on Text-to-SQL Generation
by: Wu, Cheng-Kuang, et al.
Published: (2024)

Evaluating the Efficacy of AI Techniques in Textual Anonymization: A Comparative Study
by: Asimopoulos, Dimitris, et al.
Published: (2024)

Orca: Enhancing Role-Playing Abilities of Large Language Models by Integrating Personality Traits
by: Huang, Yuxuan
Published: (2024)

RoleCraft-GLM: Advancing Personalized Role-Playing in Large Language Models
by: Tao, Meiling, et al.
Published: (2023)

Rehearse With User: Personalized Opinion Summarization via Role-Playing based on Large Language Models
by: Zhang, Yanyue, et al.
Published: (2025)

Beyond One World: Benchmarking Super Heros in Role-Playing Across Multiversal Contexts
by: Ngokpol, Perapard, et al.
Published: (2025)

ToMATO: Verbalizing the Mental States of Role-Playing LLMs for Benchmarking Theory of Mind
by: Shinoda, Kazutoshi, et al.
Published: (2025)

RPGBENCH: Evaluating Large Language Models as Role-Playing Game Engines
by: Yu, Pengfei, et al.
Published: (2025)

DynSess: Dynamic Session-Level Evaluation and Optimization Framework for Role-Playing Agents
by: Zhang, Rongsheng, et al.
Published: (2026)

Memory-Driven Role-Playing: Evaluation and Enhancement of Persona Knowledge Utilization in LLMs
by: Wang, Kai, et al.
Published: (2026)

Evaluating LLM-Generated Versus Human-Authored Responses in Role-Play Dialogues
by: Lu, Dongxu, et al.
Published: (2025)

Inflated Excellence or True Performance? Rethinking Medical Diagnostic Benchmarks with Dynamic Evaluation
by: Zhang, Xiangxu, et al.
Published: (2025)

Audio Jailbreaks in Large Audio-Language Models: Taxonomy, Attack-Defense Analysis, and Cost-Aware Evaluation
by: Feng, Bo-Han, et al.
Published: (2026)

Understanding the Role of LLMs in Multimodal Evaluation Benchmarks
by: Jiang, Botian, et al.
Published: (2024)

CoSER: A Comprehensive Literary Dataset and Framework for Training and Evaluating LLM Role-Playing and Persona Simulation
by: Wang, Xintao, et al.
Published: (2025)

The Oscars of AI Theater: A Survey on Role-Playing with Language Models
by: Chen, Nuo, et al.
Published: (2024)

Too Good to be Bad: On the Failure of LLMs to Role-Play Villains
by: Yi, Zihao, et al.
Published: (2025)

Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization
by: Tseng, Yu-Min, et al.
Published: (2024)

TraceSafe: A Systematic Assessment of LLM Guardrails on Multi-Step Tool-Calling Trajectories
by: Chen, Yen-Shan, et al.
Published: (2026)

DPRF: A Generalizable Dynamic Persona Refinement Framework for Optimizing Behavior Alignment Between Personalized LLM Role-Playing Agents and Humans
by: Yao, Bingsheng, et al.
Published: (2025)

Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction
by: Qi, Ji, et al.
Published: (2023)

Schema-Guided Culture-Aware Complex Event Simulation with Multi-Agent Role-Play
by: Li, Sha, et al.
Published: (2024)

Reasoning Does Not Necessarily Improve Role-Playing Ability
by: Feng, Xiachong, et al.
Published: (2025)

VITA-QinYu: Expressive Spoken Language Model for Role-Playing and Singing
by: Xu, Jiacheng, et al.
Published: (2026)

NoisyCausal: A Benchmark for Evaluating Causal Reasoning Under Structured Noise
by: Xu, Zhi, et al.
Published: (2026)

Answer, Refuse, or Guess? Investigating Risk-Aware Decision Making in Language Models
by: Wu, Cheng-Kuang, et al.
Published: (2025)

ASMR: Augmenting Life Scenario using Large Generative Models for Robotic Action Reflection
by: Tsai, Shang-Chi, et al.
Published: (2025)

Towards Personalized Deep Research: Benchmarks and Evaluations
by: Liang, Yuan, et al.
Published: (2025)

A Text-to-Game Engine for UGC-Based Role-Playing Games
by: Zhang, Lei, et al.
Published: (2024)

Improving LLM Reasoning through Interpretable Role-Playing Steering
by: Wang, Anyi, et al.
Published: (2025)

Playing Language Game with LLMs Leads to Jailbreaking
by: Peng, Yu, et al.
Published: (2024)