Chatbots for Data Collection in Surveys: A Comparison of Four Theory-Based Interview Probes
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915191701110784 |
|---|---|
| author | Jacobsen, Rune M. Cox, Samuel Rhys Griggio, Carla F. van Berkel, Niels |
| author_facet | Jacobsen, Rune M. Cox, Samuel Rhys Griggio, Carla F. van Berkel, Niels |
| contents | Surveys are a widespread method for collecting data at scale, but their rigid structure often limits the depth of qualitative insights obtained. While interviews naturally yield richer responses, they are challenging to conduct across diverse locations and large participant pools. To partially bridge this gap, we investigate the potential of using LLM-based chatbots to support qualitative data collection through interview probes embedded in surveys. We assess four theory-based interview probes: descriptive, idiographic, clarifying, and explanatory. Through a split-plot study design (N=64), we compare the probes' impact on response quality and user experience across three key stages of HCI research: exploration, requirements gathering, and evaluation. Our results show that probes facilitate the collection of high-quality survey data, with specific probes proving effective at different research stages. We contribute practical and methodological implications for using chatbots as research tools to enrich qualitative data collection. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_08582 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Chatbots for Data Collection in Surveys: A Comparison of Four Theory-Based Interview Probes Jacobsen, Rune M. Cox, Samuel Rhys Griggio, Carla F. van Berkel, Niels Human-Computer Interaction Surveys are a widespread method for collecting data at scale, but their rigid structure often limits the depth of qualitative insights obtained. While interviews naturally yield richer responses, they are challenging to conduct across diverse locations and large participant pools. To partially bridge this gap, we investigate the potential of using LLM-based chatbots to support qualitative data collection through interview probes embedded in surveys. We assess four theory-based interview probes: descriptive, idiographic, clarifying, and explanatory. Through a split-plot study design (N=64), we compare the probes' impact on response quality and user experience across three key stages of HCI research: exploration, requirements gathering, and evaluation. Our results show that probes facilitate the collection of high-quality survey data, with specific probes proving effective at different research stages. We contribute practical and methodological implications for using chatbots as research tools to enrich qualitative data collection. |
| title | Chatbots for Data Collection in Surveys: A Comparison of Four Theory-Based Interview Probes |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2503.08582 |