Automated Knowledge Component Generation for Interpretable Knowledge Tracing in Coding Problems
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911691182178304 |
|---|---|
| author | Duan, Zhangqi Fernandez, Nigel Narayanan, Arun Balajiee Lekshmi Hassany, Mohammad de Alencar, Rafaella Sampaio Brusilovsky, Peter Akram, Bita Lan, Andrew |
| author_facet | Duan, Zhangqi Fernandez, Nigel Narayanan, Arun Balajiee Lekshmi Hassany, Mohammad de Alencar, Rafaella Sampaio Brusilovsky, Peter Akram, Bita Lan, Andrew |
| contents | Knowledge components (KCs) mapped to problems help model student learning, tracking their mastery levels on fine-grained skills thereby facilitating personalized learning and feedback in online learning platforms. However, crafting and tagging KCs to problems, traditionally performed by human domain experts, is highly labor intensive. We present an automated, LLM-based pipeline for KC generation and tagging for open-ended programming problems. We also develop an LLM-based knowledge tracing (KT) framework to leverage these LLM-generated KCs, which we refer to as KCGen-KT. We conduct extensive quantitative and qualitative evaluations on two real-world student code submission datasets in different programming languages.We find that KCGen-KT outperforms existing KT methods and human-written KCs on future student response prediction. We investigate the learning curves of generated KCs and show that LLM-generated KCs result in a better fit than human written KCs under a cognitive model. We also conduct a human evaluation with course instructors to show that our pipeline generates reasonably accurate problem-KC mappings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_18632 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Automated Knowledge Component Generation for Interpretable Knowledge Tracing in Coding Problems Duan, Zhangqi Fernandez, Nigel Narayanan, Arun Balajiee Lekshmi Hassany, Mohammad de Alencar, Rafaella Sampaio Brusilovsky, Peter Akram, Bita Lan, Andrew Artificial Intelligence Computation and Language Computers and Society Machine Learning Software Engineering Knowledge components (KCs) mapped to problems help model student learning, tracking their mastery levels on fine-grained skills thereby facilitating personalized learning and feedback in online learning platforms. However, crafting and tagging KCs to problems, traditionally performed by human domain experts, is highly labor intensive. We present an automated, LLM-based pipeline for KC generation and tagging for open-ended programming problems. We also develop an LLM-based knowledge tracing (KT) framework to leverage these LLM-generated KCs, which we refer to as KCGen-KT. We conduct extensive quantitative and qualitative evaluations on two real-world student code submission datasets in different programming languages.We find that KCGen-KT outperforms existing KT methods and human-written KCs on future student response prediction. We investigate the learning curves of generated KCs and show that LLM-generated KCs result in a better fit than human written KCs under a cognitive model. We also conduct a human evaluation with course instructors to show that our pipeline generates reasonably accurate problem-KC mappings. |
| title | Automated Knowledge Component Generation for Interpretable Knowledge Tracing in Coding Problems |
| topic | Artificial Intelligence Computation and Language Computers and Society Machine Learning Software Engineering |
| url | https://arxiv.org/abs/2502.18632 |