Cross-Course Generalizability of SRL-Aligned Predictive Models Using Digital Learning Traces
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910163670138880 |
|---|---|
| author | Schwerter, Jakob Sabel, Loreen Bose, Judith Bernacki, Matthew L. Xu, Di Schmellenkamp, Marko Zeume, Thomas Doebler, Philipp |
| author_facet | Schwerter, Jakob Sabel, Loreen Bose, Judith Bernacki, Matthew L. Xu, Di Schmellenkamp, Marko Zeume, Thomas Doebler, Philipp |
| contents | STEM dropout rates remain high at universities, particularly in computer science programs with theory-intensive courses. Digital learning environments now capture rich behavioral data that could help identify struggling students early, yet the generalizability of data-driven prediction models across courses and institutions remains uncertain. Guided by self-regulated learning (SRL) theory, this study analyzed multimodal digital-trace data from three undergraduate theoretical computer science courses (N1 = 137, N2 = 104, N3 = 148) at two universities. Weekly SRL-aligned digital-trace indicators were modeled using Elastic Net, Random Forest, and XGBoost to evaluate predictive performance over time and across settings, and model calibration both within and across courses. Early prediction of at-risk students was feasible, with SRL-related behaviors such as time management, effort regulation, and sustained engagement emerging as key predictors. While Random Forest achieved the highest in-sample accuracy, Elastic Net generalized more robustly across contexts. Out-of-sample accuracy and calibration declined between institutions with different base rates, underscoring the contextual nature of predictive analytics in higher education. These findings suggest that digital learning traces enable early identification of at-risk students within courses, but generalizing predictive models beyond their original context requires caution, particularly if the at-risk rates differ between contexts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_22812 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Cross-Course Generalizability of SRL-Aligned Predictive Models Using Digital Learning Traces Schwerter, Jakob Sabel, Loreen Bose, Judith Bernacki, Matthew L. Xu, Di Schmellenkamp, Marko Zeume, Thomas Doebler, Philipp Computers and Society Machine Learning Applications STEM dropout rates remain high at universities, particularly in computer science programs with theory-intensive courses. Digital learning environments now capture rich behavioral data that could help identify struggling students early, yet the generalizability of data-driven prediction models across courses and institutions remains uncertain. Guided by self-regulated learning (SRL) theory, this study analyzed multimodal digital-trace data from three undergraduate theoretical computer science courses (N1 = 137, N2 = 104, N3 = 148) at two universities. Weekly SRL-aligned digital-trace indicators were modeled using Elastic Net, Random Forest, and XGBoost to evaluate predictive performance over time and across settings, and model calibration both within and across courses. Early prediction of at-risk students was feasible, with SRL-related behaviors such as time management, effort regulation, and sustained engagement emerging as key predictors. While Random Forest achieved the highest in-sample accuracy, Elastic Net generalized more robustly across contexts. Out-of-sample accuracy and calibration declined between institutions with different base rates, underscoring the contextual nature of predictive analytics in higher education. These findings suggest that digital learning traces enable early identification of at-risk students within courses, but generalizing predictive models beyond their original context requires caution, particularly if the at-risk rates differ between contexts. |
| title | Cross-Course Generalizability of SRL-Aligned Predictive Models Using Digital Learning Traces |
| topic | Computers and Society Machine Learning Applications |
| url | https://arxiv.org/abs/2604.22812 |