Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2403.06607 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909133788151808 |
|---|---|
| author | Kirsten, Elisabeth Buckmann, Annalina Mhaidli, Abraham Becker, Steffen |
| author_facet | Kirsten, Elisabeth Buckmann, Annalina Mhaidli, Abraham Becker, Steffen |
| contents | Qualitative data analysis provides insight into the underlying perceptions and experiences within unstructured data. However, the time-consuming nature of the coding process, especially for larger datasets, calls for innovative approaches, such as the integration of Large Language Models (LLMs). This short paper presents initial findings from a study investigating the integration of LLMs for coding tasks of varying complexity in a real-world dataset. Our results highlight the challenges inherent in coding with extensive codebooks and contexts, both for human coders and LLMs, and suggest that the integration of LLMs into the coding process requires a task-by-task evaluation. We examine factors influencing the complexity of coding tasks and initiate a discussion on the usefulness and limitations of incorporating LLMs in qualitative research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_06607 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Decoding Complexity: Exploring Human-AI Concordance in Qualitative Coding Kirsten, Elisabeth Buckmann, Annalina Mhaidli, Abraham Becker, Steffen Human-Computer Interaction Qualitative data analysis provides insight into the underlying perceptions and experiences within unstructured data. However, the time-consuming nature of the coding process, especially for larger datasets, calls for innovative approaches, such as the integration of Large Language Models (LLMs). This short paper presents initial findings from a study investigating the integration of LLMs for coding tasks of varying complexity in a real-world dataset. Our results highlight the challenges inherent in coding with extensive codebooks and contexts, both for human coders and LLMs, and suggest that the integration of LLMs into the coding process requires a task-by-task evaluation. We examine factors influencing the complexity of coding tasks and initiate a discussion on the usefulness and limitations of incorporating LLMs in qualitative research. |
| title | Decoding Complexity: Exploring Human-AI Concordance in Qualitative Coding |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2403.06607 |