Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhou, Xuhui, Su, Zhe, Eisape, Tiwalayo, Kim, Hyunwoo, Sap, Maarten
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866913529797279744
author	Zhou, Xuhui Su, Zhe Eisape, Tiwalayo Kim, Hyunwoo Sap, Maarten
author_facet	Zhou, Xuhui Su, Zhe Eisape, Tiwalayo Kim, Hyunwoo Sap, Maarten
contents	Recent advances in large language models (LLM) have enabled richer social simulations, allowing for the study of various social phenomena. However, most recent work has used a more omniscient perspective on these simulations (e.g., single LLM to generate all interlocutors), which is fundamentally at odds with the non-omniscient, information asymmetric interactions that involve humans and AI agents in the real world. To examine these differences, we develop an evaluation framework to simulate social interactions with LLMs in various settings (omniscient, non-omniscient). Our experiments show that LLMs perform better in unrealistic, omniscient simulation settings but struggle in ones that more accurately reflect real-world conditions with information asymmetry. Our findings indicate that addressing information asymmetry remains a fundamental challenge for LLM-based agents.
format	Preprint
id	arxiv_https___arxiv_org_abs_2403_05020
institution	arXiv
publishDate	2024
record_format	arxiv
spellingShingle	Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMs Zhou, Xuhui Su, Zhe Eisape, Tiwalayo Kim, Hyunwoo Sap, Maarten Computation and Language Artificial Intelligence Recent advances in large language models (LLM) have enabled richer social simulations, allowing for the study of various social phenomena. However, most recent work has used a more omniscient perspective on these simulations (e.g., single LLM to generate all interlocutors), which is fundamentally at odds with the non-omniscient, information asymmetric interactions that involve humans and AI agents in the real world. To examine these differences, we develop an evaluation framework to simulate social interactions with LLMs in various settings (omniscient, non-omniscient). Our experiments show that LLMs perform better in unrealistic, omniscient simulation settings but struggle in ones that more accurately reflect real-world conditions with information asymmetry. Our findings indicate that addressing information asymmetry remains a fundamental challenge for LLM-based agents.
title	Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMs
topic	Computation and Language Artificial Intelligence
url	https://arxiv.org/abs/2403.05020