Prompts Matter: Comparing ML/GAI Approaches for Generating Inductive Qualitative Coding Results
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913575448084480 |
|---|---|
| author | Chen, John Lotsos, Alexandros Zhao, Lexie Wang, Grace Wilensky, Uri Sherin, Bruce Horn, Michael |
| author_facet | Chen, John Lotsos, Alexandros Zhao, Lexie Wang, Grace Wilensky, Uri Sherin, Bruce Horn, Michael |
| contents | Inductive qualitative methods have been a mainstay of education research for decades, yet it takes much time and effort to conduct rigorously. Recent advances in artificial intelligence, particularly with generative AI (GAI), have led to initial success in generating inductive coding results. Like human coders, GAI tools rely on instructions to work, and how to instruct it may matter. To understand how ML/GAI approaches could contribute to qualitative coding processes, this study applied two known and two theory-informed novel approaches to an online community dataset and evaluated the resulting coding results. Our findings show significant discrepancies between ML/GAI approaches and demonstrate the advantage of our approaches, which introduce human coding processes into GAI prompts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_06316 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Prompts Matter: Comparing ML/GAI Approaches for Generating Inductive Qualitative Coding Results Chen, John Lotsos, Alexandros Zhao, Lexie Wang, Grace Wilensky, Uri Sherin, Bruce Horn, Michael Computation and Language Artificial Intelligence Human-Computer Interaction Inductive qualitative methods have been a mainstay of education research for decades, yet it takes much time and effort to conduct rigorously. Recent advances in artificial intelligence, particularly with generative AI (GAI), have led to initial success in generating inductive coding results. Like human coders, GAI tools rely on instructions to work, and how to instruct it may matter. To understand how ML/GAI approaches could contribute to qualitative coding processes, this study applied two known and two theory-informed novel approaches to an online community dataset and evaluated the resulting coding results. Our findings show significant discrepancies between ML/GAI approaches and demonstrate the advantage of our approaches, which introduce human coding processes into GAI prompts. |
| title | Prompts Matter: Comparing ML/GAI Approaches for Generating Inductive Qualitative Coding Results |
| topic | Computation and Language Artificial Intelligence Human-Computer Interaction |
| url | https://arxiv.org/abs/2411.06316 |