Exploring How Multiple Levels of GPT-Generated Programming Hints Support or Disappoint Novices
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914739147243520 |
|---|---|
| author | Xiao, Ruiwei Hou, Xinying Stamper, John |
| author_facet | Xiao, Ruiwei Hou, Xinying Stamper, John |
| contents | Recent studies have integrated large language models (LLMs) into diverse educational contexts, including providing adaptive programming hints, a type of feedback focuses on helping students move forward during problem-solving. However, most existing LLM-based hint systems are limited to one single hint type. To investigate whether and how different levels of hints can support students' problem-solving and learning, we conducted a think-aloud study with 12 novices using the LLM Hint Factory, a system providing four levels of hints from general natural language guidance to concrete code assistance, varying in format and granularity. We discovered that high-level natural language hints alone can be helpless or even misleading, especially when addressing next-step or syntax-related help requests. Adding lower-level hints, like code examples with in-line comments, can better support students. The findings open up future work on customizing help responses from content, format, and granularity levels to accurately identify and meet students' learning needs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_02213 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Exploring How Multiple Levels of GPT-Generated Programming Hints Support or Disappoint Novices Xiao, Ruiwei Hou, Xinying Stamper, John Human-Computer Interaction Artificial Intelligence Computers and Society Recent studies have integrated large language models (LLMs) into diverse educational contexts, including providing adaptive programming hints, a type of feedback focuses on helping students move forward during problem-solving. However, most existing LLM-based hint systems are limited to one single hint type. To investigate whether and how different levels of hints can support students' problem-solving and learning, we conducted a think-aloud study with 12 novices using the LLM Hint Factory, a system providing four levels of hints from general natural language guidance to concrete code assistance, varying in format and granularity. We discovered that high-level natural language hints alone can be helpless or even misleading, especially when addressing next-step or syntax-related help requests. Adding lower-level hints, like code examples with in-line comments, can better support students. The findings open up future work on customizing help responses from content, format, and granularity levels to accurately identify and meet students' learning needs. |
| title | Exploring How Multiple Levels of GPT-Generated Programming Hints Support or Disappoint Novices |
| topic | Human-Computer Interaction Artificial Intelligence Computers and Society |
| url | https://arxiv.org/abs/2404.02213 |