Cracking the Code: Evaluating Zero-Shot Prompting Methods for Providing Programming Feedback
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910756117676032 |
|---|---|
| author | Ippisch, Niklas Haensch, Anna-Carolina Simson, Jan Beck, Jacob Herklotz, Markus Schierholz, Malte |
| author_facet | Ippisch, Niklas Haensch, Anna-Carolina Simson, Jan Beck, Jacob Herklotz, Markus Schierholz, Malte |
| contents | Despite the growing use of large language models (LLMs) for providing feedback, limited research has explored how to achieve high-quality feedback. This case study introduces an evaluation framework to assess different zero-shot prompt engineering methods. We varied the prompts systematically and analyzed the provided feedback on programming errors in R. The results suggest that prompts suggesting a stepwise procedure increase the precision, while omitting explicit specifications about which provided data to analyze improves error identification. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_15702 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Cracking the Code: Evaluating Zero-Shot Prompting Methods for Providing Programming Feedback Ippisch, Niklas Haensch, Anna-Carolina Simson, Jan Beck, Jacob Herklotz, Markus Schierholz, Malte Software Engineering Despite the growing use of large language models (LLMs) for providing feedback, limited research has explored how to achieve high-quality feedback. This case study introduces an evaluation framework to assess different zero-shot prompt engineering methods. We varied the prompts systematically and analyzed the provided feedback on programming errors in R. The results suggest that prompts suggesting a stepwise procedure increase the precision, while omitting explicit specifications about which provided data to analyze improves error identification. |
| title | Cracking the Code: Evaluating Zero-Shot Prompting Methods for Providing Programming Feedback |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2412.15702 |