Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2405.19132 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916265181839360 |
|---|---|
| author | Scholl, Andreas Schiffner, Daniel Kiesler, Natalie |
| author_facet | Scholl, Andreas Schiffner, Daniel Kiesler, Natalie |
| contents | Large Language Models (LLMs) have taken the world by storm, and students are assumed to use related tools at a great scale. In this research paper we aim to gain an understanding of how introductory programming students chat with LLMs and related tools, e.g., ChatGPT-3.5. To address this goal, computing students at a large German university were motivated to solve programming exercises with the assistance of ChatGPT as part of their weekly introductory course exercises. Then students (n=213) submitted their chat protocols (with 2335 prompts in sum) as data basis for this analysis. The data was analyzed w.r.t. the prompts, frequencies, the chats' progress, contents, and other use pattern, which revealed a great variety of interactions, both potentially supportive and concerning. Learning about students' interactions with ChatGPT will help inform and align teaching practices and instructions for future introductory programming courses in higher education. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_19132 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Analyzing Chat Protocols of Novice Programmers Solving Introductory Programming Tasks with ChatGPT Scholl, Andreas Schiffner, Daniel Kiesler, Natalie Artificial Intelligence Large Language Models (LLMs) have taken the world by storm, and students are assumed to use related tools at a great scale. In this research paper we aim to gain an understanding of how introductory programming students chat with LLMs and related tools, e.g., ChatGPT-3.5. To address this goal, computing students at a large German university were motivated to solve programming exercises with the assistance of ChatGPT as part of their weekly introductory course exercises. Then students (n=213) submitted their chat protocols (with 2335 prompts in sum) as data basis for this analysis. The data was analyzed w.r.t. the prompts, frequencies, the chats' progress, contents, and other use pattern, which revealed a great variety of interactions, both potentially supportive and concerning. Learning about students' interactions with ChatGPT will help inform and align teaching practices and instructions for future introductory programming courses in higher education. |
| title | Analyzing Chat Protocols of Novice Programmers Solving Introductory Programming Tasks with ChatGPT |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2405.19132 |