Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929336612814848 |
|---|---|
| author | Wei, Jing Kim, Sungdong Jung, Hyunhoon Kim, Young-Ho |
| author_facet | Wei, Jing Kim, Sungdong Jung, Hyunhoon Kim, Young-Ho |
| contents | Large language models (LLMs) provide a new way to build chatbots by accepting natural language prompts. Yet, it is unclear how to design prompts to power chatbots to carry on naturalistic conversations while pursuing a given goal, such as collecting self-report data from users. We explore what design factors of prompts can help steer chatbots to talk naturally and collect data reliably. To this aim, we formulated four prompt designs with different structures and personas. Through an online study (N = 48) where participants conversed with chatbots driven by different designs of prompts, we assessed how prompt designs and conversation topics affected the conversation flows and users' perceptions of chatbots. Our chatbots covered 79% of the desired information slots during conversations, and the designs of prompts and topics significantly influenced the conversation flows and the data collection performance. We discuss the opportunities and challenges of building chatbots with LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2301_05843 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data Wei, Jing Kim, Sungdong Jung, Hyunhoon Kim, Young-Ho Human-Computer Interaction Artificial Intelligence Computation and Language H.5.2; I.2.7 Large language models (LLMs) provide a new way to build chatbots by accepting natural language prompts. Yet, it is unclear how to design prompts to power chatbots to carry on naturalistic conversations while pursuing a given goal, such as collecting self-report data from users. We explore what design factors of prompts can help steer chatbots to talk naturally and collect data reliably. To this aim, we formulated four prompt designs with different structures and personas. Through an online study (N = 48) where participants conversed with chatbots driven by different designs of prompts, we assessed how prompt designs and conversation topics affected the conversation flows and users' perceptions of chatbots. Our chatbots covered 79% of the desired information slots during conversations, and the designs of prompts and topics significantly influenced the conversation flows and the data collection performance. We discuss the opportunities and challenges of building chatbots with LLMs. |
| title | Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data |
| topic | Human-Computer Interaction Artificial Intelligence Computation and Language H.5.2; I.2.7 |
| url | https://arxiv.org/abs/2301.05843 |