MDK12-Bench: A Comprehensive Evaluation of Multimodal Large Language Models on Multidisciplinary Exams
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909730910240768 |
|---|---|
| author | Zhou, Pengfei Peng, Xiaopeng Zhang, Fanrui Xu, Zhaopan Ai, Jiaxin Qiu, Yansheng Li, Chuanhao Li, Zhen Li, Ming Feng, Yukang Sun, Jianwen Zhang, Haoquan Li, Zizhen Mao, Xiaofeng Li, Zekai Zhao, Wangbo Wang, Kai Chang, Xiaojun Shao, Wenqi You, Yang Zhang, Kaipeng |
| author_facet | Zhou, Pengfei Peng, Xiaopeng Zhang, Fanrui Xu, Zhaopan Ai, Jiaxin Qiu, Yansheng Li, Chuanhao Li, Zhen Li, Ming Feng, Yukang Sun, Jianwen Zhang, Haoquan Li, Zizhen Mao, Xiaofeng Li, Zekai Zhao, Wangbo Wang, Kai Chang, Xiaojun Shao, Wenqi You, Yang Zhang, Kaipeng |
| contents | Multimodal large language models (MLLMs), which integrate language and visual cues for problem-solving, are crucial for advancing artificial general intelligence (AGI). However, current benchmarks for measuring the intelligence of MLLMs suffer from limited scale, narrow coverage, and unstructured knowledge, offering only static and undifferentiated evaluations. To bridge this gap, we introduce MDK12-Bench, a large-scale multidisciplinary benchmark built from real-world K-12 exams spanning six disciplines with 141K instances and 6,225 knowledge points organized in a six-layer taxonomy. Covering five question formats with difficulty and year annotations, it enables comprehensive evaluation to capture the extent to which MLLMs perform over four dimensions: 1) difficulty levels, 2) temporal (cross-year) shifts, 3) contextual shifts, and 4) knowledge-driven reasoning. We propose a novel dynamic evaluation framework that introduces unfamiliar visual, textual, and question form shifts to challenge model generalization while improving benchmark objectivity and longevity by mitigating data contamination. We further evaluate knowledge-point reference-augmented generation (KP-RAG) to examine the role of knowledge in problem-solving. Key findings reveal limitations in current MLLMs in multiple aspects and provide guidance for enhancing model robustness, interpretability, and AI-assisted education. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_06851 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MDK12-Bench: A Comprehensive Evaluation of Multimodal Large Language Models on Multidisciplinary Exams Zhou, Pengfei Peng, Xiaopeng Zhang, Fanrui Xu, Zhaopan Ai, Jiaxin Qiu, Yansheng Li, Chuanhao Li, Zhen Li, Ming Feng, Yukang Sun, Jianwen Zhang, Haoquan Li, Zizhen Mao, Xiaofeng Li, Zekai Zhao, Wangbo Wang, Kai Chang, Xiaojun Shao, Wenqi You, Yang Zhang, Kaipeng Artificial Intelligence Computers and Society Multimodal large language models (MLLMs), which integrate language and visual cues for problem-solving, are crucial for advancing artificial general intelligence (AGI). However, current benchmarks for measuring the intelligence of MLLMs suffer from limited scale, narrow coverage, and unstructured knowledge, offering only static and undifferentiated evaluations. To bridge this gap, we introduce MDK12-Bench, a large-scale multidisciplinary benchmark built from real-world K-12 exams spanning six disciplines with 141K instances and 6,225 knowledge points organized in a six-layer taxonomy. Covering five question formats with difficulty and year annotations, it enables comprehensive evaluation to capture the extent to which MLLMs perform over four dimensions: 1) difficulty levels, 2) temporal (cross-year) shifts, 3) contextual shifts, and 4) knowledge-driven reasoning. We propose a novel dynamic evaluation framework that introduces unfamiliar visual, textual, and question form shifts to challenge model generalization while improving benchmark objectivity and longevity by mitigating data contamination. We further evaluate knowledge-point reference-augmented generation (KP-RAG) to examine the role of knowledge in problem-solving. Key findings reveal limitations in current MLLMs in multiple aspects and provide guidance for enhancing model robustness, interpretability, and AI-assisted education. |
| title | MDK12-Bench: A Comprehensive Evaluation of Multimodal Large Language Models on Multidisciplinary Exams |
| topic | Artificial Intelligence Computers and Society |
| url | https://arxiv.org/abs/2508.06851 |