Character is Destiny: Can Role-Playing Language Agents Make Persona-Driven Decisions?
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910701730136064 |
|---|---|
| author | Xu, Rui Wang, Xintao Chen, Jiangjie Yuan, Siyu Yuan, Xinfeng Liang, Jiaqing Chen, Zulong Dong, Xiaoqing Xiao, Yanghua |
| author_facet | Xu, Rui Wang, Xintao Chen, Jiangjie Yuan, Siyu Yuan, Xinfeng Liang, Jiaqing Chen, Zulong Dong, Xiaoqing Xiao, Yanghua |
| contents | Can Large Language Models (LLMs) simulate humans in making important decisions? Recent research has unveiled the potential of using LLMs to develop role-playing language agents (RPLAs), mimicking mainly the knowledge and tones of various characters. However, imitative decision-making necessitates a more nuanced understanding of personas. In this paper, we benchmark the ability of LLMs in persona-driven decision-making. Specifically, we investigate whether LLMs can predict characters' decisions provided by the preceding stories in high-quality novels. Leveraging character analyses written by literary experts, we construct a dataset LIFECHOICE comprising 1,462 characters' decision points from 388 books. Then, we conduct comprehensive experiments on LIFECHOICE, with various LLMs and RPLA methodologies. The results demonstrate that state-of-the-art LLMs exhibit promising capabilities in this task, yet substantial room for improvement remains. Hence, we further propose the CHARMAP method, which adopts persona-based memory retrieval and significantly advances RPLAs on this task, achieving 5.03% increase in accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_12138 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Character is Destiny: Can Role-Playing Language Agents Make Persona-Driven Decisions? Xu, Rui Wang, Xintao Chen, Jiangjie Yuan, Siyu Yuan, Xinfeng Liang, Jiaqing Chen, Zulong Dong, Xiaoqing Xiao, Yanghua Artificial Intelligence Can Large Language Models (LLMs) simulate humans in making important decisions? Recent research has unveiled the potential of using LLMs to develop role-playing language agents (RPLAs), mimicking mainly the knowledge and tones of various characters. However, imitative decision-making necessitates a more nuanced understanding of personas. In this paper, we benchmark the ability of LLMs in persona-driven decision-making. Specifically, we investigate whether LLMs can predict characters' decisions provided by the preceding stories in high-quality novels. Leveraging character analyses written by literary experts, we construct a dataset LIFECHOICE comprising 1,462 characters' decision points from 388 books. Then, we conduct comprehensive experiments on LIFECHOICE, with various LLMs and RPLA methodologies. The results demonstrate that state-of-the-art LLMs exhibit promising capabilities in this task, yet substantial room for improvement remains. Hence, we further propose the CHARMAP method, which adopts persona-based memory retrieval and significantly advances RPLAs on this task, achieving 5.03% increase in accuracy. |
| title | Character is Destiny: Can Role-Playing Language Agents Make Persona-Driven Decisions? |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2404.12138 |