Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.01150 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916717230292992 |
|---|---|
| author | Mburu, Ted K. Rong, Kangxuan McColley, Campbell J. Werth, Alexandra |
| author_facet | Mburu, Ted K. Rong, Kangxuan McColley, Campbell J. Werth, Alexandra |
| contents | This paper presents a methodological framework for using generative AI in educational survey research. We explore how Large Language Models (LLMs) can generate adaptive, context-aware survey questions and introduce the Synthetic Question-Response Analysis (SQRA) framework, which enables iterative testing and refinement of AI-generated prompts prior to deployment with human participants. Guided by Activity Theory, we analyze how AI tools mediate participant engagement and learning, and we examine ethical issues such as bias, privacy, and transparency. Through sentiment, lexical, and structural analyses of both AI-to-AI and AI-to-human survey interactions, we evaluate the alignment and effectiveness of these questions. Our findings highlight the promise and limitations of AI-driven survey instruments, emphasizing the need for robust prompt engineering and validation to support trustworthy, scalable, and contextually relevant data collection in engineering education. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_01150 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Methodological Foundations for AI-Driven Survey Question Generation Mburu, Ted K. Rong, Kangxuan McColley, Campbell J. Werth, Alexandra Computers and Society I.2.7; K.3.1 This paper presents a methodological framework for using generative AI in educational survey research. We explore how Large Language Models (LLMs) can generate adaptive, context-aware survey questions and introduce the Synthetic Question-Response Analysis (SQRA) framework, which enables iterative testing and refinement of AI-generated prompts prior to deployment with human participants. Guided by Activity Theory, we analyze how AI tools mediate participant engagement and learning, and we examine ethical issues such as bias, privacy, and transparency. Through sentiment, lexical, and structural analyses of both AI-to-AI and AI-to-human survey interactions, we evaluate the alignment and effectiveness of these questions. Our findings highlight the promise and limitations of AI-driven survey instruments, emphasizing the need for robust prompt engineering and validation to support trustworthy, scalable, and contextually relevant data collection in engineering education. |
| title | Methodological Foundations for AI-Driven Survey Question Generation |
| topic | Computers and Society I.2.7; K.3.1 |
| url | https://arxiv.org/abs/2505.01150 |