Redefining Research Crowdsourcing: Incorporating Human Feedback with LLM-Powered Digital Twins
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910975038324736 |
|---|---|
| author | Chan, Amanda Di, Catherine Rupertus, Joseph Smith, Gary Rao, Varun Nagaraj Ribeiro, Manoel Horta Monroy-Hernández, Andrés |
| author_facet | Chan, Amanda Di, Catherine Rupertus, Joseph Smith, Gary Rao, Varun Nagaraj Ribeiro, Manoel Horta Monroy-Hernández, Andrés |
| contents | Crowd work platforms like Amazon Mechanical Turk and Prolific are vital for research, yet workers' growing use of generative AI tools poses challenges. Researchers face compromised data validity as AI responses replace authentic human behavior, while workers risk diminished roles as AI automates tasks. To address this, we propose a hybrid framework using digital twins, personalized AI models that emulate workers' behaviors and preferences while keeping humans in the loop. We evaluate our system with an experiment (n=88 crowd workers) and in-depth interviews with crowd workers (n=5) and social science researchers (n=4). Our results suggest that digital twins may enhance productivity and reduce decision fatigue while maintaining response quality. Both researchers and workers emphasized the importance of transparency, ethical data use, and worker agency. By automating repetitive tasks and preserving human engagement for nuanced ones, digital twins may help balance scalability with authenticity. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_24004 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Redefining Research Crowdsourcing: Incorporating Human Feedback with LLM-Powered Digital Twins Chan, Amanda Di, Catherine Rupertus, Joseph Smith, Gary Rao, Varun Nagaraj Ribeiro, Manoel Horta Monroy-Hernández, Andrés Human-Computer Interaction Computation and Language Computers and Society Crowd work platforms like Amazon Mechanical Turk and Prolific are vital for research, yet workers' growing use of generative AI tools poses challenges. Researchers face compromised data validity as AI responses replace authentic human behavior, while workers risk diminished roles as AI automates tasks. To address this, we propose a hybrid framework using digital twins, personalized AI models that emulate workers' behaviors and preferences while keeping humans in the loop. We evaluate our system with an experiment (n=88 crowd workers) and in-depth interviews with crowd workers (n=5) and social science researchers (n=4). Our results suggest that digital twins may enhance productivity and reduce decision fatigue while maintaining response quality. Both researchers and workers emphasized the importance of transparency, ethical data use, and worker agency. By automating repetitive tasks and preserving human engagement for nuanced ones, digital twins may help balance scalability with authenticity. |
| title | Redefining Research Crowdsourcing: Incorporating Human Feedback with LLM-Powered Digital Twins |
| topic | Human-Computer Interaction Computation and Language Computers and Society |
| url | https://arxiv.org/abs/2505.24004 |