Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2504.02234 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909638273794048 |
|---|---|
| author | Anthis, Jacy Reese Liu, Ryan Richardson, Sean M. Kozlowski, Austin C. Koch, Bernard Evans, James Brynjolfsson, Erik Bernstein, Michael |
| author_facet | Anthis, Jacy Reese Liu, Ryan Richardson, Sean M. Kozlowski, Austin C. Koch, Bernard Evans, James Brynjolfsson, Erik Bernstein, Michael |
| contents | Accurate and verifiable large language model (LLM) simulations of human research subjects promise an accessible data source for understanding human behavior and training new AI systems. However, results to date have been limited, and few social scientists have adopted this method. In this position paper, we argue that the promise of LLM social simulations can be achieved by addressing five tractable challenges. We ground our argument in a review of empirical comparisons between LLMs and human research subjects, commentaries on the topic, and related work. We identify promising directions, including context-rich prompting and fine-tuning with social science datasets. We believe that LLM social simulations can already be used for pilot and exploratory studies, and more widespread use may soon be possible with rapidly advancing LLM capabilities. Researchers should prioritize developing conceptual models and iterative evaluations to make the best use of new AI systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_02234 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LLM Social Simulations Are a Promising Research Method Anthis, Jacy Reese Liu, Ryan Richardson, Sean M. Kozlowski, Austin C. Koch, Bernard Evans, James Brynjolfsson, Erik Bernstein, Michael Human-Computer Interaction Artificial Intelligence Computation and Language Computers and Society Accurate and verifiable large language model (LLM) simulations of human research subjects promise an accessible data source for understanding human behavior and training new AI systems. However, results to date have been limited, and few social scientists have adopted this method. In this position paper, we argue that the promise of LLM social simulations can be achieved by addressing five tractable challenges. We ground our argument in a review of empirical comparisons between LLMs and human research subjects, commentaries on the topic, and related work. We identify promising directions, including context-rich prompting and fine-tuning with social science datasets. We believe that LLM social simulations can already be used for pilot and exploratory studies, and more widespread use may soon be possible with rapidly advancing LLM capabilities. Researchers should prioritize developing conceptual models and iterative evaluations to make the best use of new AI systems. |
| title | LLM Social Simulations Are a Promising Research Method |
| topic | Human-Computer Interaction Artificial Intelligence Computation and Language Computers and Society |
| url | https://arxiv.org/abs/2504.02234 |