Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2407.06576 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915999093096448 |
|---|---|
| author | Moon, Suhong Abdulhai, Marwa Kang, Minwoo Suh, Joseph Soedarmadji, Widyadewi Behar, Eran Kohen Chan, David M. Canny, John |
| author_facet | Moon, Suhong Abdulhai, Marwa Kang, Minwoo Suh, Joseph Soedarmadji, Widyadewi Behar, Eran Kohen Chan, David M. Canny, John |
| contents | Large language models (LLMs) are trained from vast repositories of text authored by millions of distinct authors, reflecting an enormous diversity of human traits. While these models bear the potential to be used as approximations of human subjects in behavioral studies, prior efforts have been limited in steering model responses to match individual human users. In this work, we introduce "Anthology", a method for conditioning LLMs to particular virtual personas by harnessing open-ended life narratives, which we refer to as "backstories." We show that our methodology enhances the consistency and reliability of experimental outcomes while ensuring better representation of diverse sub-populations. Across three nationally representative human surveys conducted as part of Pew Research Center's American Trends Panel (ATP), we demonstrate that Anthology achieves up to 18% improvement in matching the response distributions of human respondents and 27% improvement in consistency metrics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_06576 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Virtual Personas for Language Models via an Anthology of Backstories Moon, Suhong Abdulhai, Marwa Kang, Minwoo Suh, Joseph Soedarmadji, Widyadewi Behar, Eran Kohen Chan, David M. Canny, John Computation and Language Artificial Intelligence Large language models (LLMs) are trained from vast repositories of text authored by millions of distinct authors, reflecting an enormous diversity of human traits. While these models bear the potential to be used as approximations of human subjects in behavioral studies, prior efforts have been limited in steering model responses to match individual human users. In this work, we introduce "Anthology", a method for conditioning LLMs to particular virtual personas by harnessing open-ended life narratives, which we refer to as "backstories." We show that our methodology enhances the consistency and reliability of experimental outcomes while ensuring better representation of diverse sub-populations. Across three nationally representative human surveys conducted as part of Pew Research Center's American Trends Panel (ATP), we demonstrate that Anthology achieves up to 18% improvement in matching the response distributions of human respondents and 27% improvement in consistency metrics. |
| title | Virtual Personas for Language Models via an Anthology of Backstories |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2407.06576 |