From Solver to Tutor: Evaluating the Pedagogical Intelligence of LLMs with KMP-Bench
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908862041292800 |
|---|---|
| author | Shi, Weikang Ren, Houxing Pan, Junting Zhou, Aojun Wang, Ke Lu, Zimu Yang, Yunqiao Hu, Yuxuan Wei, Linda Zhan, Mingjie Li, Hongsheng |
| author_facet | Shi, Weikang Ren, Houxing Pan, Junting Zhou, Aojun Wang, Ke Lu, Zimu Yang, Yunqiao Hu, Yuxuan Wei, Linda Zhan, Mingjie Li, Hongsheng |
| contents | Large Language Models (LLMs) show significant potential in AI mathematical tutoring, yet current evaluations often rely on simplistic metrics or narrow pedagogical scenarios, failing to assess comprehensive, multi-turn teaching effectiveness. In this paper, we introduce KMP-Bench, a comprehensive K-8 Mathematical Pedagogical Benchmark designed to assess LLMs from two complementary perspectives. The first module, KMP-Dialogue, evaluates holistic pedagogical capabilities against six core principles (e.g., Challenge, Explanation, Feedback), leveraging a novel multi-turn dialogue dataset constructed by weaving together diverse pedagogical components. The second module, KMP-Skills, provides a granular assessment of foundational tutoring abilities, including multi-turn problem-solving, error detection and correction, and problem generation. Our evaluations on KMP-Bench reveal a key disparity: while leading LLMs excel at tasks with verifiable solutions, they struggle with the nuanced application of pedagogical principles. Additionally, we present KMP-Pile, a large-scale (150K) dialogue dataset. Models fine-tuned on KMP-Pile show substantial improvement on KMP-Bench, underscoring the value of pedagogically-rich training data for developing more effective AI math tutors. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_02775 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | From Solver to Tutor: Evaluating the Pedagogical Intelligence of LLMs with KMP-Bench Shi, Weikang Ren, Houxing Pan, Junting Zhou, Aojun Wang, Ke Lu, Zimu Yang, Yunqiao Hu, Yuxuan Wei, Linda Zhan, Mingjie Li, Hongsheng Computation and Language Machine Learning Large Language Models (LLMs) show significant potential in AI mathematical tutoring, yet current evaluations often rely on simplistic metrics or narrow pedagogical scenarios, failing to assess comprehensive, multi-turn teaching effectiveness. In this paper, we introduce KMP-Bench, a comprehensive K-8 Mathematical Pedagogical Benchmark designed to assess LLMs from two complementary perspectives. The first module, KMP-Dialogue, evaluates holistic pedagogical capabilities against six core principles (e.g., Challenge, Explanation, Feedback), leveraging a novel multi-turn dialogue dataset constructed by weaving together diverse pedagogical components. The second module, KMP-Skills, provides a granular assessment of foundational tutoring abilities, including multi-turn problem-solving, error detection and correction, and problem generation. Our evaluations on KMP-Bench reveal a key disparity: while leading LLMs excel at tasks with verifiable solutions, they struggle with the nuanced application of pedagogical principles. Additionally, we present KMP-Pile, a large-scale (150K) dialogue dataset. Models fine-tuned on KMP-Pile show substantial improvement on KMP-Bench, underscoring the value of pedagogically-rich training data for developing more effective AI math tutors. |
| title | From Solver to Tutor: Evaluating the Pedagogical Intelligence of LLMs with KMP-Bench |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2603.02775 |