How Far Are We? The Triumphs and Trials of Generative AI in Learning Software Engineering
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909128434122752 |
|---|---|
| author | Choudhuri, Rudrajit Liu, Dylan Steinmacher, Igor Gerosa, Marco Sarma, Anita |
| author_facet | Choudhuri, Rudrajit Liu, Dylan Steinmacher, Igor Gerosa, Marco Sarma, Anita |
| contents | Conversational Generative AI (convo-genAI) is revolutionizing Software Engineering (SE) as engineers and academics embrace this technology in their work. However, there is a gap in understanding the current potential and pitfalls of this technology, specifically in supporting students in SE tasks. In this work, we evaluate through a between-subjects study (N=22) the effectiveness of ChatGPT, a convo-genAI platform, in assisting students in SE tasks. Our study did not find statistical differences in participants' productivity or self-efficacy when using ChatGPT as compared to traditional resources, but we found significantly increased frustration levels. Our study also revealed 5 distinct faults arising from violations of Human-AI interaction guidelines, which led to 7 different (negative) consequences on participants. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_11719 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | How Far Are We? The Triumphs and Trials of Generative AI in Learning Software Engineering Choudhuri, Rudrajit Liu, Dylan Steinmacher, Igor Gerosa, Marco Sarma, Anita Software Engineering Human-Computer Interaction Conversational Generative AI (convo-genAI) is revolutionizing Software Engineering (SE) as engineers and academics embrace this technology in their work. However, there is a gap in understanding the current potential and pitfalls of this technology, specifically in supporting students in SE tasks. In this work, we evaluate through a between-subjects study (N=22) the effectiveness of ChatGPT, a convo-genAI platform, in assisting students in SE tasks. Our study did not find statistical differences in participants' productivity or self-efficacy when using ChatGPT as compared to traditional resources, but we found significantly increased frustration levels. Our study also revealed 5 distinct faults arising from violations of Human-AI interaction guidelines, which led to 7 different (negative) consequences on participants. |
| title | How Far Are We? The Triumphs and Trials of Generative AI in Learning Software Engineering |
| topic | Software Engineering Human-Computer Interaction |
| url | https://arxiv.org/abs/2312.11719 |