Evaluating Vision-Language and Large Language Models for Automated Student Assessment in Indonesian Classrooms
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917375545180160 |
|---|---|
| author | Aisyah, Nurul Kautsar, Muhammad Dehan Al Hidayat, Arif Chowdhury, Raqib Koto, Fajri |
| author_facet | Aisyah, Nurul Kautsar, Muhammad Dehan Al Hidayat, Arif Chowdhury, Raqib Koto, Fajri |
| contents | Despite rapid progress in vision-language and large language models (VLMs and LLMs), their effectiveness for AI-driven educational assessment in real-world, underrepresented classrooms remains largely unexplored. We evaluate state-of-the-art VLMs and LLMs on over 14K handwritten answers from grade-4 classrooms in Indonesia, covering Mathematics and English aligned with the local national curriculum. Unlike prior work on clean digital text, our dataset features naturally curly, diverse handwriting from real classrooms, posing realistic visual and linguistic challenges. Assessment tasks include grading and generating personalized Indonesian feedback guided by rubric-based evaluation. Results show that the VLM struggles with handwriting recognition, causing error propagation in LLM grading, yet LLM feedback remains pedagogically useful despite imperfect visual inputs, revealing limits in personalization and contextual relevance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_04822 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Evaluating Vision-Language and Large Language Models for Automated Student Assessment in Indonesian Classrooms Aisyah, Nurul Kautsar, Muhammad Dehan Al Hidayat, Arif Chowdhury, Raqib Koto, Fajri Computation and Language Despite rapid progress in vision-language and large language models (VLMs and LLMs), their effectiveness for AI-driven educational assessment in real-world, underrepresented classrooms remains largely unexplored. We evaluate state-of-the-art VLMs and LLMs on over 14K handwritten answers from grade-4 classrooms in Indonesia, covering Mathematics and English aligned with the local national curriculum. Unlike prior work on clean digital text, our dataset features naturally curly, diverse handwriting from real classrooms, posing realistic visual and linguistic challenges. Assessment tasks include grading and generating personalized Indonesian feedback guided by rubric-based evaluation. Results show that the VLM struggles with handwriting recognition, causing error propagation in LLM grading, yet LLM feedback remains pedagogically useful despite imperfect visual inputs, revealing limits in personalization and contextual relevance. |
| title | Evaluating Vision-Language and Large Language Models for Automated Student Assessment in Indonesian Classrooms |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2506.04822 |