LLMs cannot spot math errors, even when allowed to peek into the solution
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909763948773376 |
|---|---|
| author | Srivatsa, KV Aditya Maurya, Kaushal Kumar Kochmar, Ekaterina |
| author_facet | Srivatsa, KV Aditya Maurya, Kaushal Kumar Kochmar, Ekaterina |
| contents | Large language models (LLMs) demonstrate remarkable performance on math word problems, yet they have been shown to struggle with meta-reasoning tasks such as identifying errors in student solutions. In this work, we investigate the challenge of locating the first error step in stepwise solutions using two error reasoning datasets: VtG and PRM800K. Our experiments show that state-of-the-art LLMs struggle to locate the first error step in student solutions even when given access to the reference solution. To that end, we propose an approach that generates an intermediate corrected student solution, aligning more closely with the original student's solution, which helps improve performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_01395 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LLMs cannot spot math errors, even when allowed to peek into the solution Srivatsa, KV Aditya Maurya, Kaushal Kumar Kochmar, Ekaterina Computation and Language Artificial Intelligence Large language models (LLMs) demonstrate remarkable performance on math word problems, yet they have been shown to struggle with meta-reasoning tasks such as identifying errors in student solutions. In this work, we investigate the challenge of locating the first error step in stepwise solutions using two error reasoning datasets: VtG and PRM800K. Our experiments show that state-of-the-art LLMs struggle to locate the first error step in student solutions even when given access to the reference solution. To that end, we propose an approach that generates an intermediate corrected student solution, aligning more closely with the original student's solution, which helps improve performance. |
| title | LLMs cannot spot math errors, even when allowed to peek into the solution |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2509.01395 |