Human-in-the-Loop LLM Grading for Handwritten Mathematics Assessments
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912964991254528 |
|---|---|
| author | Vanhoyweghen, Arne Holst, Vincent Mobini, Melika Van de Voorde, Lukas Vanleke, Tibo Verbruggen, Bert Verbeken, Brecht Algaba, Andres Verboven, Sam Guerry, Marie-Anne Van Droogenbroeck, Filip Ginis, Vincent |
| author_facet | Vanhoyweghen, Arne Holst, Vincent Mobini, Melika Van de Voorde, Lukas Vanleke, Tibo Verbruggen, Bert Verbeken, Brecht Algaba, Andres Verboven, Sam Guerry, Marie-Anne Van Droogenbroeck, Filip Ginis, Vincent |
| contents | Providing timely and individualised feedback on handwritten student work is highly beneficial for learning but difficult to achieve at scale. This challenge has become more pressing as generative AI undermines the reliability of take-home assessments, shifting emphasis toward supervised, in-class evaluation. We present a scalable, end-to-end workflow for LLM-assisted grading of short, pen-and-paper assessments. The workflow spans (1) constructing solution keys, (2) developing detailed rubric-style grading keys used to guide the LLM, and (3) a grading procedure that combines automated scanning and anonymisation, multi-pass LLM scoring, automated consistency checks, and mandatory human verification. We deploy the system in two undergraduate mathematics courses using six low-stakes in-class tests. Empirically, LLM assistance reduces grading time by approximately 23% while achieving agreement comparable to, and in several cases tighter than, fully manual grading. Occasional model errors occur but are effectively contained by the hybrid design. Overall, our results show that carefully embedded human-in-the-loop LLM grading can substantially reduce workload while maintaining fairness and accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_13083 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Human-in-the-Loop LLM Grading for Handwritten Mathematics Assessments Vanhoyweghen, Arne Holst, Vincent Mobini, Melika Van de Voorde, Lukas Vanleke, Tibo Verbruggen, Bert Verbeken, Brecht Algaba, Andres Verboven, Sam Guerry, Marie-Anne Van Droogenbroeck, Filip Ginis, Vincent Computers and Society Artificial Intelligence Providing timely and individualised feedback on handwritten student work is highly beneficial for learning but difficult to achieve at scale. This challenge has become more pressing as generative AI undermines the reliability of take-home assessments, shifting emphasis toward supervised, in-class evaluation. We present a scalable, end-to-end workflow for LLM-assisted grading of short, pen-and-paper assessments. The workflow spans (1) constructing solution keys, (2) developing detailed rubric-style grading keys used to guide the LLM, and (3) a grading procedure that combines automated scanning and anonymisation, multi-pass LLM scoring, automated consistency checks, and mandatory human verification. We deploy the system in two undergraduate mathematics courses using six low-stakes in-class tests. Empirically, LLM assistance reduces grading time by approximately 23% while achieving agreement comparable to, and in several cases tighter than, fully manual grading. Occasional model errors occur but are effectively contained by the hybrid design. Overall, our results show that carefully embedded human-in-the-loop LLM grading can substantially reduce workload while maintaining fairness and accuracy. |
| title | Human-in-the-Loop LLM Grading for Handwritten Mathematics Assessments |
| topic | Computers and Society Artificial Intelligence |
| url | https://arxiv.org/abs/2603.13083 |