AI Literacy as a Key Driver of User Experience in AI-Powered Assessment: Insights from Socratic Mind
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912508516761600 |
|---|---|
| author | Soylu, Meryem Yilmaz Lee, Jeonghyun Hung, Jui-Tse Cui, Christopher Zhang Joyner, David A. |
| author_facet | Soylu, Meryem Yilmaz Lee, Jeonghyun Hung, Jui-Tse Cui, Christopher Zhang Joyner, David A. |
| contents | As Artificial Intelligence (AI) tools become increasingly embedded in higher education, understanding how students interact with these systems is essential to supporting effective learning. This study examines how students' AI literacy and prior exposure to AI technologies shape their perceptions of Socratic Mind, an interactive AI-powered formative assessment tool. Drawing on Self-Determination Theory and user experience research, we analyze relationships among AI literacy, perceived usability, satisfaction, engagement, and perceived learning effectiveness. Data from 309 undergraduates in Computer Science and Business courses were collected through validated surveys. Partial least squares structural equation modeling showed that AI literacy - especially self-efficacy, conceptual understanding, and application skills - significantly predicts usability, satisfaction, and engagement. Usability and satisfaction, in turn, strongly predict perceived learning effectiveness, while prior AI exposure showed no significant effect. These findings highlight that AI literacy, rather than exposure alone, shapes student experiences. Designers should integrate adaptive guidance and user-centered features to support diverse literacy levels, fostering inclusive, motivating, and effective AI-based learning environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_21654 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AI Literacy as a Key Driver of User Experience in AI-Powered Assessment: Insights from Socratic Mind Soylu, Meryem Yilmaz Lee, Jeonghyun Hung, Jui-Tse Cui, Christopher Zhang Joyner, David A. Human-Computer Interaction Artificial Intelligence K.3.1; I.2.6 As Artificial Intelligence (AI) tools become increasingly embedded in higher education, understanding how students interact with these systems is essential to supporting effective learning. This study examines how students' AI literacy and prior exposure to AI technologies shape their perceptions of Socratic Mind, an interactive AI-powered formative assessment tool. Drawing on Self-Determination Theory and user experience research, we analyze relationships among AI literacy, perceived usability, satisfaction, engagement, and perceived learning effectiveness. Data from 309 undergraduates in Computer Science and Business courses were collected through validated surveys. Partial least squares structural equation modeling showed that AI literacy - especially self-efficacy, conceptual understanding, and application skills - significantly predicts usability, satisfaction, and engagement. Usability and satisfaction, in turn, strongly predict perceived learning effectiveness, while prior AI exposure showed no significant effect. These findings highlight that AI literacy, rather than exposure alone, shapes student experiences. Designers should integrate adaptive guidance and user-centered features to support diverse literacy levels, fostering inclusive, motivating, and effective AI-based learning environments. |
| title | AI Literacy as a Key Driver of User Experience in AI-Powered Assessment: Insights from Socratic Mind |
| topic | Human-Computer Interaction Artificial Intelligence K.3.1; I.2.6 |
| url | https://arxiv.org/abs/2507.21654 |