Examining the Use and Impact of an AI Code Assistant on Developer Productivity and Experience in the Enterprise
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929741085278208 |
|---|---|
| author | Weisz, Justin D. Kumar, Shraddha Muller, Michael Browne, Karen-Ellen Goldberg, Arielle Heintze, Ellice Bajpai, Shagun |
| author_facet | Weisz, Justin D. Kumar, Shraddha Muller, Michael Browne, Karen-Ellen Goldberg, Arielle Heintze, Ellice Bajpai, Shagun |
| contents | AI assistants are being created to help software engineers conduct a variety of coding-related tasks, such as writing, documenting, and testing code. We describe the use of the watsonx Code Assistant (WCA), an LLM-powered coding assistant deployed internally within IBM. Through surveys of two user cohorts (N=669) and unmoderated usability testing (N=15), we examined developers' experiences with WCA and its impact on their productivity. We learned about their motivations for using (or not using) WCA, we examined their expectations of its speed and quality, and we identified new considerations regarding ownership of and responsibility for generated code. Our case study characterizes the impact of an LLM-powered assistant on developers' perceptions of productivity and it shows that although such tools do often provide net productivity increases, these benefits may not always be experienced by all users. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_06603 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Examining the Use and Impact of an AI Code Assistant on Developer Productivity and Experience in the Enterprise Weisz, Justin D. Kumar, Shraddha Muller, Michael Browne, Karen-Ellen Goldberg, Arielle Heintze, Ellice Bajpai, Shagun Human-Computer Interaction Software Engineering AI assistants are being created to help software engineers conduct a variety of coding-related tasks, such as writing, documenting, and testing code. We describe the use of the watsonx Code Assistant (WCA), an LLM-powered coding assistant deployed internally within IBM. Through surveys of two user cohorts (N=669) and unmoderated usability testing (N=15), we examined developers' experiences with WCA and its impact on their productivity. We learned about their motivations for using (or not using) WCA, we examined their expectations of its speed and quality, and we identified new considerations regarding ownership of and responsibility for generated code. Our case study characterizes the impact of an LLM-powered assistant on developers' perceptions of productivity and it shows that although such tools do often provide net productivity increases, these benefits may not always be experienced by all users. |
| title | Examining the Use and Impact of an AI Code Assistant on Developer Productivity and Experience in the Enterprise |
| topic | Human-Computer Interaction Software Engineering |
| url | https://arxiv.org/abs/2412.06603 |