Automated Distractor and Feedback Generation for Math Multiple-choice Questions via In-context Learning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911755090788352 |
|---|---|
| author | McNichols, Hunter Feng, Wanyong Lee, Jaewook Scarlatos, Alexander Smith, Digory Woodhead, Simon Lan, Andrew |
| author_facet | McNichols, Hunter Feng, Wanyong Lee, Jaewook Scarlatos, Alexander Smith, Digory Woodhead, Simon Lan, Andrew |
| contents | Multiple-choice questions (MCQs) are ubiquitous in almost all levels of education since they are easy to administer, grade, and are a reliable form of assessment. An important aspect of MCQs is the distractors, i.e., incorrect options that are designed to target specific misconceptions or insufficient knowledge among students. To date, the task of crafting high-quality distractors has largely remained a labor-intensive process for teachers and learning content designers, which has limited scalability. In this work, we explore the task of automated distractor and corresponding feedback message generation in math MCQs using large language models. We establish a formulation of these two tasks and propose a simple, in-context learning-based solution. Moreover, we propose generative AI-based metrics for evaluating the quality of the feedback messages. We conduct extensive experiments on these tasks using a real-world MCQ dataset. Our findings suggest that there is a lot of room for improvement in automated distractor and feedback generation; based on these findings, we outline several directions for future work. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2308_03234 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Automated Distractor and Feedback Generation for Math Multiple-choice Questions via In-context Learning McNichols, Hunter Feng, Wanyong Lee, Jaewook Scarlatos, Alexander Smith, Digory Woodhead, Simon Lan, Andrew Computation and Language Multiple-choice questions (MCQs) are ubiquitous in almost all levels of education since they are easy to administer, grade, and are a reliable form of assessment. An important aspect of MCQs is the distractors, i.e., incorrect options that are designed to target specific misconceptions or insufficient knowledge among students. To date, the task of crafting high-quality distractors has largely remained a labor-intensive process for teachers and learning content designers, which has limited scalability. In this work, we explore the task of automated distractor and corresponding feedback message generation in math MCQs using large language models. We establish a formulation of these two tasks and propose a simple, in-context learning-based solution. Moreover, we propose generative AI-based metrics for evaluating the quality of the feedback messages. We conduct extensive experiments on these tasks using a real-world MCQ dataset. Our findings suggest that there is a lot of room for improvement in automated distractor and feedback generation; based on these findings, we outline several directions for future work. |
| title | Automated Distractor and Feedback Generation for Math Multiple-choice Questions via In-context Learning |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2308.03234 |