Cover Image

SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviors

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hu, Tiancheng, Baumann, Joachim, Lupo, Lorenzo, Collier, Nigel, Hovy, Dirk, Röttger, Paul
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Computers and Society Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Quantifying the Persona Effect in LLM Simulations
by: Hu, Tiancheng, et al.
Published: (2024)

iNews: A Multimodal Dataset for Modeling Personalized Affective Responses to News
by: Hu, Tiancheng, et al.
Published: (2025)

Can LLM be a Personalized Judge?
by: Dong, Yijiang River, et al.
Published: (2024)

Towards Human-Level Text Coding with LLMs: The Case of Fatherhood Roles in Public Policy Documents
by: Lupo, Lorenzo, et al.
Published: (2023)

Compromesso! Italian Many-Shot Jailbreaks Undermine the Safety of Large Language Models
by: Pernisi, Fabio, et al.
Published: (2024)

The Pluralistic Moral Gap: Understanding Judgment and Value Differences between Humans and Large Language Models
by: Russo, Giuseppe, et al.
Published: (2025)

Generative Language Models Exhibit Social Identity Biases
by: Hu, Tiancheng, et al.
Published: (2023)

The Ecological Fallacy in Annotation: Modelling Human Label Variation goes beyond Sociodemographics
by: Orlikowski, Matthias, et al.
Published: (2023)

SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety
by: Röttger, Paul, et al.
Published: (2024)

JiraiBench: A Bilingual Benchmark for Evaluating Large Language Models' Detection of Human Self-Destructive Behavior Content in Jirai Community
by: Xiao, Yunze, et al.
Published: (2025)

Impoverished Language Technology: The Lack of (Social) Class in NLP
by: Curry, Amanda Cercas, et al.
Published: (2024)

Fairer Preferences Elicit Improved Human-Aligned Large Language Model Judgments
by: Zhou, Han, et al.
Published: (2024)

Beyond Flesch-Kincaid: Prompt-based Metrics Improve Difficulty Classification of Educational Texts
by: Rooein, Donya, et al.
Published: (2024)

Divine LLaMAs: Bias, Stereotypes, Stigmatization, and Emotion Representation of Religion in Large Language Models
by: Plaza-del-Arco, Flor Miriam, et al.
Published: (2024)

TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Law
by: Hui, Zheng, et al.
Published: (2025)

DADIT: A Dataset for Demographic Classification of Italian Twitter Users and a Comparison of Prediction Methods
by: Lupo, Lorenzo, et al.
Published: (2024)

Large Language Model Hacking: Quantifying the Hidden Risks of Using LLMs for Text Annotation
by: Baumann, Joachim, et al.
Published: (2025)

IssueBench: Millions of Realistic Prompts for Measuring Issue Bias in LLM Writing Assistance
by: Röttger, Paul, et al.
Published: (2025)

Beyond Demographics: Fine-tuning Large Language Models to Predict Individuals' Subjective Text Perceptions
by: Orlikowski, Matthias, et al.
Published: (2025)

DarkBench: Benchmarking Dark Patterns in Large Language Models
by: Kran, Esben, et al.
Published: (2025)

XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models
by: Röttger, Paul, et al.
Published: (2023)

How Far Are LLMs from Believable AI? A Benchmark for Evaluating the Believability of Human Behavior Simulation
by: Xiao, Yang, et al.
Published: (2023)

ElectionSim: Massive Population Election Simulation Powered by Large Language Model Driven Agents
by: Zhang, Xinnong, et al.
Published: (2024)

Can LLMs Think Like Consumers? Benchmarking Crowd-Level Reaction Reconstruction with ConsumerSimBench
by: Wang, Tianyu, et al.
Published: (2026)

Can Language Models Recognize Convincing Arguments?
by: Rescala, Paula, et al.
Published: (2024)

CBT-Bench: Evaluating Large Language Models on Assisting Cognitive Behavior Therapy
by: Zhang, Mian, et al.
Published: (2024)

WorldValuesBench: A Large-Scale Benchmark Dataset for Multi-Cultural Value Awareness of Language Models
by: Zhao, Wenlong, et al.
Published: (2024)

Navigating the Alignment-Calibration Trade-off: A Pareto-Superior Frontier via Model Merging
by: Hu, Tiancheng, et al.
Published: (2025)

Fine-Grained Behavior Simulation with Role-Playing Large Language Model on Social Media
by: Li, Kun, et al.
Published: (2024)

Diffusion Language Models Are Natively Length-Aware
by: Rossi, Vittorio, et al.
Published: (2026)

Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance
by: de Araujo, Pedro Henrique Luz, et al.
Published: (2025)

Can Large Language Models Replace Human Coders? Introducing ContentBench
by: Haman, Michael
Published: (2026)

Human vs. Machine: Behavioral Differences Between Expert Humans and Language Models in Wargame Simulations
by: Lamparth, Max, et al.
Published: (2024)

Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models
by: Röttger, Paul, et al.
Published: (2024)

Human-Level and Beyond: Benchmarking Large Language Models Against Clinical Pharmacists in Prescription Review
by: Yang, Yan, et al.
Published: (2025)

LocalValueBench: A Collaboratively Built and Extensible Benchmark for Evaluating Localized Value Alignment and Ethical Safety in Large Language Models
by: Meadows, Gwenyth Isobel, et al.
Published: (2024)

Beyond the Final Layer: Intermediate Representations for Better Multilingual Calibration in Large Language Models
by: Zhou, Ej, et al.
Published: (2025)

Steer Model beyond Assistant: Controlling System Prompt Strength via Contrastive Decoding
by: Dong, Yijiang River, et al.
Published: (2026)

No for Some, Yes for Others: Persona Prompts and Other Sources of False Refusal in Language Models
by: Plaza-del-Arco, Flor Miriam, et al.
Published: (2025)

HugAgent: Benchmarking LLMs for Simulation of Individualized Human Reasoning
by: Li, Chance Jiajie, et al.
Published: (2025)