Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.15473 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910786550497280 |
|---|---|
| author | Gao, Ge Leon, Amelia Jetten, Andrea Turner, Jasmine Almoubayyed, Husni Fancsali, Stephen Brunskill, Emma |
| author_facet | Gao, Ge Leon, Amelia Jetten, Andrea Turner, Jasmine Almoubayyed, Husni Fancsali, Stephen Brunskill, Emma |
| contents | Educational stakeholders are often particularly interested in sparse, delayed student outcomes, like end-of-year statewide exams. The rare occurrence of such assessments makes it harder to identify students likely to fail such assessments, as well as making it slow for researchers and educators to be able to assess the effectiveness of particular educational tools. Prior work has primarily focused on using logs from students full usage (e.g. year-long) of an educational product to predict outcomes, or considered predictive accuracy using a few minutes to predict outcomes after a short (e.g. 1 hour) session. In contrast, we investigate machine learning predictors using students' logs during their first few hours of usage can provide useful predictive insight into those students' end-of-school year external assessment. We do this on three diverse datasets: from students in Uganda using a literacy game product, and from students in the US using two mathematics intelligent tutoring systems. We consider various measures of the accuracy of the resulting predictors, including its ability to identify students at different parts along the assessment performance distribution. Our findings suggest that short-term log usage data, from 2-5 hours, can be used to provide valuable signal about students' long-term external performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_15473 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Predicting Long-Term Student Outcomes from Short-Term EdTech Log Data Gao, Ge Leon, Amelia Jetten, Andrea Turner, Jasmine Almoubayyed, Husni Fancsali, Stephen Brunskill, Emma Computers and Society Human-Computer Interaction Machine Learning Educational stakeholders are often particularly interested in sparse, delayed student outcomes, like end-of-year statewide exams. The rare occurrence of such assessments makes it harder to identify students likely to fail such assessments, as well as making it slow for researchers and educators to be able to assess the effectiveness of particular educational tools. Prior work has primarily focused on using logs from students full usage (e.g. year-long) of an educational product to predict outcomes, or considered predictive accuracy using a few minutes to predict outcomes after a short (e.g. 1 hour) session. In contrast, we investigate machine learning predictors using students' logs during their first few hours of usage can provide useful predictive insight into those students' end-of-school year external assessment. We do this on three diverse datasets: from students in Uganda using a literacy game product, and from students in the US using two mathematics intelligent tutoring systems. We consider various measures of the accuracy of the resulting predictors, including its ability to identify students at different parts along the assessment performance distribution. Our findings suggest that short-term log usage data, from 2-5 hours, can be used to provide valuable signal about students' long-term external performance. |
| title | Predicting Long-Term Student Outcomes from Short-Term EdTech Log Data |
| topic | Computers and Society Human-Computer Interaction Machine Learning |
| url | https://arxiv.org/abs/2412.15473 |