Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.23660 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916039598538752 |
|---|---|
| author | Donaldson, Jonah R. Navaz, Aliya Doran, Konstantinos Lim, Alysta Campanelli, Mario |
| author_facet | Donaldson, Jonah R. Navaz, Aliya Doran, Konstantinos Lim, Alysta Campanelli, Mario |
| contents | The rapid advancement of Large Language Models (LLMs) has introduced new possibilities and challenges in physics education, necessitating rigorous evaluation of their capabilities as both problem solvers and automated assessors. This paper presents the results of three complementary studies that evaluated frontier models released between mid-2024 and late-2025. Models were assessed on their ability to generate accurate, step-by-step solutions to university-level physics problems in Classical Mechanics, Electromagnetism, and Quantum Mechanics, and subsequently on their reliability in grading student solutions against a formal mark scheme. The results indicate a clear trajectory toward benchmark saturation in text-based reasoning, with recent architectures (such as ChatGPT-5.1 and Gemini 3.0 Pro) achieving near-perfect scores. Furthermore, recent advances in native multimodal integration have resolved previous limitations in spatial geometry and topological interpretation, enabling models to accurately process accompanying diagrams. As automated assessors, newer models demonstrated significant improvements in alignment with human grading, heavily mitigating the systemic over-marking observed in earlier iterations. However, while models reliably evaluate fully correct handwritten work, assigning partial credit to flawed or incomplete reasoning remains a persistent challenge. These findings suggest that as of late 2025, LLMs offer viable support for both independent student learning and instructional automation, provided their limitations in evaluating ambiguous reasoning are actively managed. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_23660 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Using Large Language Models in Physics Education Donaldson, Jonah R. Navaz, Aliya Doran, Konstantinos Lim, Alysta Campanelli, Mario Physics Education The rapid advancement of Large Language Models (LLMs) has introduced new possibilities and challenges in physics education, necessitating rigorous evaluation of their capabilities as both problem solvers and automated assessors. This paper presents the results of three complementary studies that evaluated frontier models released between mid-2024 and late-2025. Models were assessed on their ability to generate accurate, step-by-step solutions to university-level physics problems in Classical Mechanics, Electromagnetism, and Quantum Mechanics, and subsequently on their reliability in grading student solutions against a formal mark scheme. The results indicate a clear trajectory toward benchmark saturation in text-based reasoning, with recent architectures (such as ChatGPT-5.1 and Gemini 3.0 Pro) achieving near-perfect scores. Furthermore, recent advances in native multimodal integration have resolved previous limitations in spatial geometry and topological interpretation, enabling models to accurately process accompanying diagrams. As automated assessors, newer models demonstrated significant improvements in alignment with human grading, heavily mitigating the systemic over-marking observed in earlier iterations. However, while models reliably evaluate fully correct handwritten work, assigning partial credit to flawed or incomplete reasoning remains a persistent challenge. These findings suggest that as of late 2025, LLMs offer viable support for both independent student learning and instructional automation, provided their limitations in evaluating ambiguous reasoning are actively managed. |
| title | Using Large Language Models in Physics Education |
| topic | Physics Education |
| url | https://arxiv.org/abs/2605.23660 |