Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2504.00241 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908292595318784 |
|---|---|
| author | Karanjai, Rabimba Shor, Boris Austin, Amanda Kennedy, Ryan Lu, Yang Xu, Lei Shi, Weidong |
| author_facet | Karanjai, Rabimba Shor, Boris Austin, Amanda Kennedy, Ryan Lu, Yang Xu, Lei Shi, Weidong |
| contents | This paper investigates the use of Large Language Models (LLMs) to synthesize public opinion data, addressing challenges in traditional survey methods like declining response rates and non-response bias. We introduce a novel technique: role creation based on knowledge injection, a form of in-context learning that leverages RAG and specified personality profiles from the HEXACO model and demographic information, and uses that for dynamically generated prompts. This method allows LLMs to simulate diverse opinions more accurately than existing prompt engineering approaches. We compare our results with pre-trained models with standard few-shot prompts. Experiments using questions from the Cooperative Election Study (CES) demonstrate that our role-creation approach significantly improves the alignment of LLM-generated opinions with real-world human survey responses, increasing answer adherence. In addition, we discuss challenges, limitations and future research directions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_00241 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Synthesizing Public Opinions with LLMs: Role Creation, Impacts, and the Future to eDemorcacy Karanjai, Rabimba Shor, Boris Austin, Amanda Kennedy, Ryan Lu, Yang Xu, Lei Shi, Weidong Computation and Language Artificial Intelligence This paper investigates the use of Large Language Models (LLMs) to synthesize public opinion data, addressing challenges in traditional survey methods like declining response rates and non-response bias. We introduce a novel technique: role creation based on knowledge injection, a form of in-context learning that leverages RAG and specified personality profiles from the HEXACO model and demographic information, and uses that for dynamically generated prompts. This method allows LLMs to simulate diverse opinions more accurately than existing prompt engineering approaches. We compare our results with pre-trained models with standard few-shot prompts. Experiments using questions from the Cooperative Election Study (CES) demonstrate that our role-creation approach significantly improves the alignment of LLM-generated opinions with real-world human survey responses, increasing answer adherence. In addition, we discuss challenges, limitations and future research directions. |
| title | Synthesizing Public Opinions with LLMs: Role Creation, Impacts, and the Future to eDemorcacy |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2504.00241 |