Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929322223206400 |
|---|---|
| author | Mozannar, Hussein Bansal, Gagan Fourney, Adam Horvitz, Eric |
| author_facet | Mozannar, Hussein Bansal, Gagan Fourney, Adam Horvitz, Eric |
| contents | Code-recommendation systems, such as Copilot and CodeWhisperer, have the potential to improve programmer productivity by suggesting and auto-completing code. However, to fully realize their potential, we must understand how programmers interact with these systems and identify ways to improve that interaction. To seek insights about human-AI collaboration with code recommendations systems, we studied GitHub Copilot, a code-recommendation system used by millions of programmers daily. We developed CUPS, a taxonomy of common programmer activities when interacting with Copilot. Our study of 21 programmers, who completed coding tasks and retrospectively labeled their sessions with CUPS, showed that CUPS can help us understand how programmers interact with code-recommendation systems, revealing inefficiencies and time costs. Our insights reveal how programmers interact with Copilot and motivate new interface designs and metrics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2210_14306 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming Mozannar, Hussein Bansal, Gagan Fourney, Adam Horvitz, Eric Software Engineering Human-Computer Interaction Machine Learning Code-recommendation systems, such as Copilot and CodeWhisperer, have the potential to improve programmer productivity by suggesting and auto-completing code. However, to fully realize their potential, we must understand how programmers interact with these systems and identify ways to improve that interaction. To seek insights about human-AI collaboration with code recommendations systems, we studied GitHub Copilot, a code-recommendation system used by millions of programmers daily. We developed CUPS, a taxonomy of common programmer activities when interacting with Copilot. Our study of 21 programmers, who completed coding tasks and retrospectively labeled their sessions with CUPS, showed that CUPS can help us understand how programmers interact with code-recommendation systems, revealing inefficiencies and time costs. Our insights reveal how programmers interact with Copilot and motivate new interface designs and metrics. |
| title | Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming |
| topic | Software Engineering Human-Computer Interaction Machine Learning |
| url | https://arxiv.org/abs/2210.14306 |