MathDoc: Benchmarking Structured Extraction and Active Refusal on Noisy Mathematics Exam Papers
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918290318688256 |
|---|---|
| author | Zhou, Chenyue Tuo, Jiayi Qin, Shitong Dai, Wei Wang, Mingxuan Zhao, Ziwei Li, Duoyang Su, Shiyang Lu, Yanxi Ma, Yanbiao |
| author_facet | Zhou, Chenyue Tuo, Jiayi Qin, Shitong Dai, Wei Wang, Mingxuan Zhao, Ziwei Li, Duoyang Su, Shiyang Lu, Yanxi Ma, Yanbiao |
| contents | The automated extraction of structured questions from paper-based mathematics exams is fundamental to intelligent education, yet remains challenging in real-world settings due to severe visual noise. Existing benchmarks mainly focus on clean documents or generic layout analysis, overlooking both the structural integrity of mathematical problems and the ability of models to actively reject incomplete inputs. We introduce MathDoc, the first benchmark for document-level information extraction from authentic high school mathematics exam papers. MathDoc contains \textbf{3,609} carefully curated questions with real-world artifacts and explicitly includes unrecognizable samples to evaluate active refusal behavior. We propose a multi-dimensional evaluation framework covering stem accuracy, visual similarity, and refusal capability. Experiments on SOTA MLLMs, including Qwen3-VL and Gemini-2.5-Pro, show that although end-to-end models achieve strong extraction performance, they consistently fail to refuse illegible inputs, instead producing confident but invalid outputs. These results highlight a critical gap in current MLLMs and establish MathDoc as a benchmark for assessing model reliability under degraded document conditions. Our project repository is available at \href{https://github.com/winnk123/papers/tree/master}{GitHub repository} |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_10104 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | MathDoc: Benchmarking Structured Extraction and Active Refusal on Noisy Mathematics Exam Papers Zhou, Chenyue Tuo, Jiayi Qin, Shitong Dai, Wei Wang, Mingxuan Zhao, Ziwei Li, Duoyang Su, Shiyang Lu, Yanxi Ma, Yanbiao Computer Vision and Pattern Recognition Artificial Intelligence The automated extraction of structured questions from paper-based mathematics exams is fundamental to intelligent education, yet remains challenging in real-world settings due to severe visual noise. Existing benchmarks mainly focus on clean documents or generic layout analysis, overlooking both the structural integrity of mathematical problems and the ability of models to actively reject incomplete inputs. We introduce MathDoc, the first benchmark for document-level information extraction from authentic high school mathematics exam papers. MathDoc contains \textbf{3,609} carefully curated questions with real-world artifacts and explicitly includes unrecognizable samples to evaluate active refusal behavior. We propose a multi-dimensional evaluation framework covering stem accuracy, visual similarity, and refusal capability. Experiments on SOTA MLLMs, including Qwen3-VL and Gemini-2.5-Pro, show that although end-to-end models achieve strong extraction performance, they consistently fail to refuse illegible inputs, instead producing confident but invalid outputs. These results highlight a critical gap in current MLLMs and establish MathDoc as a benchmark for assessing model reliability under degraded document conditions. Our project repository is available at \href{https://github.com/winnk123/papers/tree/master}{GitHub repository} |
| title | MathDoc: Benchmarking Structured Extraction and Active Refusal on Noisy Mathematics Exam Papers |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2601.10104 |