LecEval: An Automated Metric for Multimodal Knowledge Acquisition in Multimedia Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913819735883776 |
|---|---|
| author | Yin, Joy Lim Jia Zhang-Li, Daniel Yu, Jifan Li, Haoxuan Tu, Shangqing Wang, Yuanchun Liu, Zhiyuan Liu, Huiqin Hou, Lei Li, Juanzi Xu, Bin |
| author_facet | Yin, Joy Lim Jia Zhang-Li, Daniel Yu, Jifan Li, Haoxuan Tu, Shangqing Wang, Yuanchun Liu, Zhiyuan Liu, Huiqin Hou, Lei Li, Juanzi Xu, Bin |
| contents | Evaluating the quality of slide-based multimedia instruction is challenging. Existing methods like manual assessment, reference-based metrics, and large language model evaluators face limitations in scalability, context capture, or bias. In this paper, we introduce LecEval, an automated metric grounded in Mayer's Cognitive Theory of Multimedia Learning, to evaluate multimodal knowledge acquisition in slide-based learning. LecEval assesses effectiveness using four rubrics: Content Relevance (CR), Expressive Clarity (EC), Logical Structure (LS), and Audience Engagement (AE). We curate a large-scale dataset of over 2,000 slides from more than 50 online course videos, annotated with fine-grained human ratings across these rubrics. A model trained on this dataset demonstrates superior accuracy and adaptability compared to existing metrics, bridging the gap between automated and human assessments. We release our dataset and toolkits at https://github.com/JoylimJY/LecEval. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_02078 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LecEval: An Automated Metric for Multimodal Knowledge Acquisition in Multimedia Learning Yin, Joy Lim Jia Zhang-Li, Daniel Yu, Jifan Li, Haoxuan Tu, Shangqing Wang, Yuanchun Liu, Zhiyuan Liu, Huiqin Hou, Lei Li, Juanzi Xu, Bin Computation and Language Artificial Intelligence Evaluating the quality of slide-based multimedia instruction is challenging. Existing methods like manual assessment, reference-based metrics, and large language model evaluators face limitations in scalability, context capture, or bias. In this paper, we introduce LecEval, an automated metric grounded in Mayer's Cognitive Theory of Multimedia Learning, to evaluate multimodal knowledge acquisition in slide-based learning. LecEval assesses effectiveness using four rubrics: Content Relevance (CR), Expressive Clarity (EC), Logical Structure (LS), and Audience Engagement (AE). We curate a large-scale dataset of over 2,000 slides from more than 50 online course videos, annotated with fine-grained human ratings across these rubrics. A model trained on this dataset demonstrates superior accuracy and adaptability compared to existing metrics, bridging the gap between automated and human assessments. We release our dataset and toolkits at https://github.com/JoylimJY/LecEval. |
| title | LecEval: An Automated Metric for Multimodal Knowledge Acquisition in Multimedia Learning |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2505.02078 |