Saved in:
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2508.20244 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912557551321088 |
|---|---|
| author | Zheng, Jiayu Hao, Lingxin Lu, Kelun Garg, Ashi Reese, Mike Yap, Melo-Jean Wang, I-Jeng Wu, Xingyun Huang, Wenrui Hoffman, Jenna Kelly, Ariane Le, My Zhang, Ryan Lin, Yanyu Faayez, Muhammad Liu, Anqi |
| author_facet | Zheng, Jiayu Hao, Lingxin Lu, Kelun Garg, Ashi Reese, Mike Yap, Melo-Jean Wang, I-Jeng Wu, Xingyun Huang, Wenrui Hoffman, Jenna Kelly, Ariane Le, My Zhang, Ryan Lin, Yanyu Faayez, Muhammad Liu, Anqi |
| contents | This study explores how college students interact with generative AI (ChatGPT-4) during educational quizzes, focusing on reliance and predictors of AI adoption. Conducted at the early stages of ChatGPT implementation, when students had limited familiarity with the tool, this field study analyzed 315 student-AI conversations during a brief, quiz-based scenario across various STEM courses. A novel four-stage reliance taxonomy was introduced to capture students' reliance patterns, distinguishing AI competence, relevance, adoption, and students' final answer correctness. Three findings emerged. First, students exhibited overall low reliance on AI and many of them could not effectively use AI for learning. Second, negative reliance patterns often persisted across interactions, highlighting students' difficulty in effectively shifting strategies after unsuccessful initial experiences. Third, certain behavioral metrics strongly predicted AI reliance, highlighting potential behavioral mechanisms to explain AI adoption. The study's findings underline critical implications for ethical AI integration in education and the broader field. It emphasizes the need for enhanced onboarding processes to improve student's familiarity and effective use of AI tools. Furthermore, AI interfaces should be designed with reliance-calibration mechanisms to enhance appropriate reliance. Ultimately, this research advances understanding of AI reliance dynamics, providing foundational insights for ethically sound and cognitively enriching AI practices. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_20244 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Do Students Rely on AI? Analysis of Student-ChatGPT Conversations from a Field Study Zheng, Jiayu Hao, Lingxin Lu, Kelun Garg, Ashi Reese, Mike Yap, Melo-Jean Wang, I-Jeng Wu, Xingyun Huang, Wenrui Hoffman, Jenna Kelly, Ariane Le, My Zhang, Ryan Lin, Yanyu Faayez, Muhammad Liu, Anqi Artificial Intelligence This study explores how college students interact with generative AI (ChatGPT-4) during educational quizzes, focusing on reliance and predictors of AI adoption. Conducted at the early stages of ChatGPT implementation, when students had limited familiarity with the tool, this field study analyzed 315 student-AI conversations during a brief, quiz-based scenario across various STEM courses. A novel four-stage reliance taxonomy was introduced to capture students' reliance patterns, distinguishing AI competence, relevance, adoption, and students' final answer correctness. Three findings emerged. First, students exhibited overall low reliance on AI and many of them could not effectively use AI for learning. Second, negative reliance patterns often persisted across interactions, highlighting students' difficulty in effectively shifting strategies after unsuccessful initial experiences. Third, certain behavioral metrics strongly predicted AI reliance, highlighting potential behavioral mechanisms to explain AI adoption. The study's findings underline critical implications for ethical AI integration in education and the broader field. It emphasizes the need for enhanced onboarding processes to improve student's familiarity and effective use of AI tools. Furthermore, AI interfaces should be designed with reliance-calibration mechanisms to enhance appropriate reliance. Ultimately, this research advances understanding of AI reliance dynamics, providing foundational insights for ethically sound and cognitively enriching AI practices. |
| title | Do Students Rely on AI? Analysis of Student-ChatGPT Conversations from a Field Study |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2508.20244 |