The Wisdom of Partisan Crowds: Comparing Collective Intelligence in Humans and LLM-based Agents
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911778056699904 |
|---|---|
| author | Chuang, Yun-Shiuan Suresh, Siddharth Harlalka, Nikunj Goyal, Agam Hawkins, Robert Yang, Sijia Shah, Dhavan Hu, Junjie Rogers, Timothy T. |
| author_facet | Chuang, Yun-Shiuan Suresh, Siddharth Harlalka, Nikunj Goyal, Agam Hawkins, Robert Yang, Sijia Shah, Dhavan Hu, Junjie Rogers, Timothy T. |
| contents | Human groups are able to converge on more accurate beliefs through deliberation, even in the presence of polarization and partisan bias -- a phenomenon known as the "wisdom of partisan crowds." Generated agents powered by Large Language Models (LLMs) are increasingly used to simulate human collective behavior, yet few benchmarks exist for evaluating their dynamics against the behavior of human groups. In this paper, we examine the extent to which the wisdom of partisan crowds emerges in groups of LLM-based agents that are prompted to role-play as partisan personas (e.g., Democrat or Republican). We find that they not only display human-like partisan biases, but also converge to more accurate beliefs through deliberation as humans do. We then identify several factors that interfere with convergence, including the use of chain-of-thought prompt and lack of details in personas. Conversely, fine-tuning on human data appears to enhance convergence. These findings show the potential and limitations of LLM-based agents as a model of human collective intelligence. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_09665 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | The Wisdom of Partisan Crowds: Comparing Collective Intelligence in Humans and LLM-based Agents Chuang, Yun-Shiuan Suresh, Siddharth Harlalka, Nikunj Goyal, Agam Hawkins, Robert Yang, Sijia Shah, Dhavan Hu, Junjie Rogers, Timothy T. Computation and Language Human groups are able to converge on more accurate beliefs through deliberation, even in the presence of polarization and partisan bias -- a phenomenon known as the "wisdom of partisan crowds." Generated agents powered by Large Language Models (LLMs) are increasingly used to simulate human collective behavior, yet few benchmarks exist for evaluating their dynamics against the behavior of human groups. In this paper, we examine the extent to which the wisdom of partisan crowds emerges in groups of LLM-based agents that are prompted to role-play as partisan personas (e.g., Democrat or Republican). We find that they not only display human-like partisan biases, but also converge to more accurate beliefs through deliberation as humans do. We then identify several factors that interfere with convergence, including the use of chain-of-thought prompt and lack of details in personas. Conversely, fine-tuning on human data appears to enhance convergence. These findings show the potential and limitations of LLM-based agents as a model of human collective intelligence. |
| title | The Wisdom of Partisan Crowds: Comparing Collective Intelligence in Humans and LLM-based Agents |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2311.09665 |