Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2504.00408 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912302334214144 |
|---|---|
| author | Weijers, Ruben Wu, Denton Betts, Hannah Jacod, Tamara Guan, Yuxiang Sujaya, Vidya Dev, Kushal Goel, Toshali Delooze, William Rabbany, Reihaneh Wu, Ying Godbout, Jean-François Pelrine, Kellin |
| author_facet | Weijers, Ruben Wu, Denton Betts, Hannah Jacod, Tamara Guan, Yuxiang Sujaya, Vidya Dev, Kushal Goel, Toshali Delooze, William Rabbany, Reihaneh Wu, Ying Godbout, Jean-François Pelrine, Kellin |
| contents | Generative AI has the potential to transform personalization and accessibility of education. However, it raises serious concerns about accuracy and helping students become independent critical thinkers. In this study, we designed a helpful AI "Peer" to help students correct fundamental physics misconceptions related to Newtonian mechanic concepts. In contrast to approaches that seek near-perfect accuracy to create an authoritative AI tutor or teacher, we directly inform students that this AI can answer up to 40% of questions incorrectly. In a randomized controlled trial with 165 students, those who engaged in targeted dialogue with the AI Peer achieved post-test scores that were, on average, 10.5 percentage points higher - with over 20 percentage points higher normalized gain - than a control group that discussed physics history. Qualitative feedback indicated that 91% of the treatment group's AI interactions were rated as helpful. Furthermore, by comparing student performance on pre- and post-test questions about the same concept, along with experts' annotations of the AI interactions, we find initial evidence suggesting the improvement in performance does not depend on the correctness of the AI. With further research, the AI Peer paradigm described here could open new possibilities for how we learn, adapt to, and grow with AI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_00408 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | From Intuition to Understanding: Using AI Peers to Overcome Physics Misconceptions Weijers, Ruben Wu, Denton Betts, Hannah Jacod, Tamara Guan, Yuxiang Sujaya, Vidya Dev, Kushal Goel, Toshali Delooze, William Rabbany, Reihaneh Wu, Ying Godbout, Jean-François Pelrine, Kellin Computers and Society Artificial Intelligence Human-Computer Interaction Generative AI has the potential to transform personalization and accessibility of education. However, it raises serious concerns about accuracy and helping students become independent critical thinkers. In this study, we designed a helpful AI "Peer" to help students correct fundamental physics misconceptions related to Newtonian mechanic concepts. In contrast to approaches that seek near-perfect accuracy to create an authoritative AI tutor or teacher, we directly inform students that this AI can answer up to 40% of questions incorrectly. In a randomized controlled trial with 165 students, those who engaged in targeted dialogue with the AI Peer achieved post-test scores that were, on average, 10.5 percentage points higher - with over 20 percentage points higher normalized gain - than a control group that discussed physics history. Qualitative feedback indicated that 91% of the treatment group's AI interactions were rated as helpful. Furthermore, by comparing student performance on pre- and post-test questions about the same concept, along with experts' annotations of the AI interactions, we find initial evidence suggesting the improvement in performance does not depend on the correctness of the AI. With further research, the AI Peer paradigm described here could open new possibilities for how we learn, adapt to, and grow with AI. |
| title | From Intuition to Understanding: Using AI Peers to Overcome Physics Misconceptions |
| topic | Computers and Society Artificial Intelligence Human-Computer Interaction |
| url | https://arxiv.org/abs/2504.00408 |