Rethinking Code Review Workflows with LLM Assistance: An Empirical Study
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916751406530560 |
|---|---|
| author | Aðalsteinsson, Fannar Steinn Magnússon, Björn Borgar Milicevic, Mislav Davidsson, Adam Nirving Cheng, Chih-Hong |
| author_facet | Aðalsteinsson, Fannar Steinn Magnússon, Björn Borgar Milicevic, Mislav Davidsson, Adam Nirving Cheng, Chih-Hong |
| contents | Code reviews are a critical yet time-consuming aspect of modern software development, increasingly challenged by growing system complexity and the demand for faster delivery. This paper presents a study conducted at WirelessCar Sweden AB, combining an exploratory field study of current code review practices with a field experiment involving two variations of an LLM-assisted code review tool. The field study identifies key challenges in traditional code reviews, including frequent context switching, insufficient contextual information, and highlights both opportunities (e.g., automatic summarization of complex pull requests) and concerns (e.g., false positives and trust issues) in using LLMs. In the field experiment, we developed two prototype variations: one offering LLM-generated reviews upfront and the other enabling on-demand interaction. Both utilize a semantic search pipeline based on retrieval-augmented generation to assemble relevant contextual information for the review, thereby tackling the uncovered challenges. Developers evaluated both variations in real-world settings: AI-led reviews are overall more preferred, while still being conditional on the reviewers' familiarity with the code base, as well as on the severity of the pull request. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_16339 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Rethinking Code Review Workflows with LLM Assistance: An Empirical Study Aðalsteinsson, Fannar Steinn Magnússon, Björn Borgar Milicevic, Mislav Davidsson, Adam Nirving Cheng, Chih-Hong Software Engineering Code reviews are a critical yet time-consuming aspect of modern software development, increasingly challenged by growing system complexity and the demand for faster delivery. This paper presents a study conducted at WirelessCar Sweden AB, combining an exploratory field study of current code review practices with a field experiment involving two variations of an LLM-assisted code review tool. The field study identifies key challenges in traditional code reviews, including frequent context switching, insufficient contextual information, and highlights both opportunities (e.g., automatic summarization of complex pull requests) and concerns (e.g., false positives and trust issues) in using LLMs. In the field experiment, we developed two prototype variations: one offering LLM-generated reviews upfront and the other enabling on-demand interaction. Both utilize a semantic search pipeline based on retrieval-augmented generation to assemble relevant contextual information for the review, thereby tackling the uncovered challenges. Developers evaluated both variations in real-world settings: AI-led reviews are overall more preferred, while still being conditional on the reviewers' familiarity with the code base, as well as on the severity of the pull request. |
| title | Rethinking Code Review Workflows with LLM Assistance: An Empirical Study |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2505.16339 |