Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.17056 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913073107828736 |
|---|---|
| author | Tang, Luoxi Sundar, Tharunya Meng, Yuqiao Yang, Shuai Patra, Ankita Chippada, Lakshmi Manohar Zhao, Jiqian Li, Yi Ma, Weicheng Xi, Zhaohan |
| author_facet | Tang, Luoxi Sundar, Tharunya Meng, Yuqiao Yang, Shuai Patra, Ankita Chippada, Lakshmi Manohar Zhao, Jiqian Li, Yi Ma, Weicheng Xi, Zhaohan |
| contents | As large language models (LLMs) are increasingly integrated into educational tools, current evaluations on standardized tests predominantly focus on binary outcome accuracy. Instead, an effective AI tutor must exhibit faithful reasoning, elucidate solution strategies, and diagnose specific human misconceptions. To bridge this gap, we introduce a pedagogical diagnostic framework that models English Standardized Test (EST) problem-solving as a traversal through a cognitive framework. Based on this framework, we present ESTBook, a multimodal benchmark encompassing 10,576 questions and 29 task types across five major exams. Unlike traditional datasets, ESTBook goes beyond data aggregation by enriching questions with formalized reasoning trajectories and distractor rationales that capture specific cognitive traps. Through extensive evaluations, we empirically demonstrate the practical utility of our diagnostic framework, showing that identifying cognitive trajectories facilitates the mitigation of performance gap and improves pedagogical reasoning through guided elicitation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_17056 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | From Test-taking to Cognitive Scaffolding: A Pedagogical Diagnostic Benchmark for LLMs on English Standardized Tests Tang, Luoxi Sundar, Tharunya Meng, Yuqiao Yang, Shuai Patra, Ankita Chippada, Lakshmi Manohar Zhao, Jiqian Li, Yi Ma, Weicheng Xi, Zhaohan Computation and Language Artificial Intelligence As large language models (LLMs) are increasingly integrated into educational tools, current evaluations on standardized tests predominantly focus on binary outcome accuracy. Instead, an effective AI tutor must exhibit faithful reasoning, elucidate solution strategies, and diagnose specific human misconceptions. To bridge this gap, we introduce a pedagogical diagnostic framework that models English Standardized Test (EST) problem-solving as a traversal through a cognitive framework. Based on this framework, we present ESTBook, a multimodal benchmark encompassing 10,576 questions and 29 task types across five major exams. Unlike traditional datasets, ESTBook goes beyond data aggregation by enriching questions with formalized reasoning trajectories and distractor rationales that capture specific cognitive traps. Through extensive evaluations, we empirically demonstrate the practical utility of our diagnostic framework, showing that identifying cognitive trajectories facilitates the mitigation of performance gap and improves pedagogical reasoning through guided elicitation. |
| title | From Test-taking to Cognitive Scaffolding: A Pedagogical Diagnostic Benchmark for LLMs on English Standardized Tests |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2505.17056 |