Exploring the Human-LLM Synergy in Advancing Theory-driven Qualitative Analysis
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918219018665984 |
|---|---|
| author | Meng, Han Yang, Yitian Fu, Wayne Lee, Jungup Li, Yunan Lee, Yi-Chieh |
| author_facet | Meng, Han Yang, Yitian Fu, Wayne Lee, Jungup Li, Yunan Lee, Yi-Chieh |
| contents | Qualitative coding is a demanding yet crucial research method in the field of Human-Computer Interaction (HCI). While recent studies have shown the capability of large language models (LLMs) to perform qualitative coding within theoretical frameworks, their potential for collaborative human-LLM discovery and generation of new insights beyond initial theory remains underexplored. To bridge this gap, we proposed CHALET, a novel approach that harnesses the power of human-LLM partnership to advance theory-driven qualitative analysis by facilitating iterative coding, disagreement analysis, and conceptualization of qualitative data. We demonstrated CHALET's utility by applying it to the qualitative analysis of conversations related to mental-illness stigma, using the attribution model as the theoretical framework. Results highlighted the unique contribution of human-LLM collaboration in uncovering latent themes of stigma across the cognitive, emotional, and behavioral dimensions. We discuss the methodological implications of the human-LLM collaborative approach to theory-based qualitative analysis for the HCI community and beyond. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_05758 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Exploring the Human-LLM Synergy in Advancing Theory-driven Qualitative Analysis Meng, Han Yang, Yitian Fu, Wayne Lee, Jungup Li, Yunan Lee, Yi-Chieh Human-Computer Interaction Computation and Language Computers and Society Qualitative coding is a demanding yet crucial research method in the field of Human-Computer Interaction (HCI). While recent studies have shown the capability of large language models (LLMs) to perform qualitative coding within theoretical frameworks, their potential for collaborative human-LLM discovery and generation of new insights beyond initial theory remains underexplored. To bridge this gap, we proposed CHALET, a novel approach that harnesses the power of human-LLM partnership to advance theory-driven qualitative analysis by facilitating iterative coding, disagreement analysis, and conceptualization of qualitative data. We demonstrated CHALET's utility by applying it to the qualitative analysis of conversations related to mental-illness stigma, using the attribution model as the theoretical framework. Results highlighted the unique contribution of human-LLM collaboration in uncovering latent themes of stigma across the cognitive, emotional, and behavioral dimensions. We discuss the methodological implications of the human-LLM collaborative approach to theory-based qualitative analysis for the HCI community and beyond. |
| title | Exploring the Human-LLM Synergy in Advancing Theory-driven Qualitative Analysis |
| topic | Human-Computer Interaction Computation and Language Computers and Society |
| url | https://arxiv.org/abs/2405.05758 |