Effects of a Prompt Engineering Intervention on Undergraduate Students' AI Self-Efficacy, AI Knowledge and Prompt Engineering Ability: A Mixed Methods Study
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917136055664640 |
|---|---|
| author | Woo, David James Wang, Deliang Yung, Tim Guo, Kai |
| author_facet | Woo, David James Wang, Deliang Yung, Tim Guo, Kai |
| contents | Prompt engineering is critical for effective interaction with large language models (LLMs) such as ChatGPT. However, efforts to teach this skill to students have been limited. This study designed and implemented a prompt engineering intervention, examining its influence on undergraduate students' AI self-efficacy, AI knowledge, and proficiency in creating effective prompts. The intervention involved 27 students who participated in a 100-minute workshop conducted during their history course at a university in Hong Kong. During the workshop, students were introduced to prompt engineering strategies, which they applied to plan the course's final essay task. Multiple data sources were collected, including students' responses to pre- and post-workshop questionnaires, pre- and post-workshop prompt libraries, and written reflections. The study's findings revealed that students demonstrated a higher level of AI self-efficacy, an enhanced understanding of AI concepts, and improved prompt engineering skills because of the intervention. These findings have implications for AI literacy education, as they highlight the importance of prompt engineering training for specific higher education use cases. This is a significant shift from students haphazardly and intuitively learning to engineer prompts. Through prompt engineering education, educators can faciitate students' effective navigation and leverage of LLMs to support their coursework. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_07302 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Effects of a Prompt Engineering Intervention on Undergraduate Students' AI Self-Efficacy, AI Knowledge and Prompt Engineering Ability: A Mixed Methods Study Woo, David James Wang, Deliang Yung, Tim Guo, Kai Computers and Society Computation and Language Human-Computer Interaction Prompt engineering is critical for effective interaction with large language models (LLMs) such as ChatGPT. However, efforts to teach this skill to students have been limited. This study designed and implemented a prompt engineering intervention, examining its influence on undergraduate students' AI self-efficacy, AI knowledge, and proficiency in creating effective prompts. The intervention involved 27 students who participated in a 100-minute workshop conducted during their history course at a university in Hong Kong. During the workshop, students were introduced to prompt engineering strategies, which they applied to plan the course's final essay task. Multiple data sources were collected, including students' responses to pre- and post-workshop questionnaires, pre- and post-workshop prompt libraries, and written reflections. The study's findings revealed that students demonstrated a higher level of AI self-efficacy, an enhanced understanding of AI concepts, and improved prompt engineering skills because of the intervention. These findings have implications for AI literacy education, as they highlight the importance of prompt engineering training for specific higher education use cases. This is a significant shift from students haphazardly and intuitively learning to engineer prompts. Through prompt engineering education, educators can faciitate students' effective navigation and leverage of LLMs to support their coursework. |
| title | Effects of a Prompt Engineering Intervention on Undergraduate Students' AI Self-Efficacy, AI Knowledge and Prompt Engineering Ability: A Mixed Methods Study |
| topic | Computers and Society Computation and Language Human-Computer Interaction |
| url | https://arxiv.org/abs/2408.07302 |