ChatGPT: Friend or Foe When Comprehending and Changing Unfamiliar Code
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917480963768320 |
|---|---|
| author | Anderson, Norman Alakmeh, Tarek Jackson, Victoria Pereira, Guilherme Vaz Akirmak, Umit Estey, Anthony Prikladnicki, Rafael Fritz, Thomas van der Hoek, André Storey, Margaret-Anne |
| author_facet | Anderson, Norman Alakmeh, Tarek Jackson, Victoria Pereira, Guilherme Vaz Akirmak, Umit Estey, Anthony Prikladnicki, Rafael Fritz, Thomas van der Hoek, André Storey, Margaret-Anne |
| contents | A rapidly growing body of research is examining how LLMs influence developers when they code. To date, this research has tended to focus on productivity and code quality outcomes, rather than the underlying cognitive processes involved in programming. To address this gap, we report on the results of an exploratory laboratory study of ten advanced student developers (five with support from AI and five without) who had to make a non-trivial extension to a sizable software system. Leveraging Polya's four problem-solving phases and 25 inductively-generated codes detailing distinct problem-solving behaviors as the primary lenses, we examined: (1) how AI impacted the problem-solving approach the developers used to solve the programming task, and (2) how AI impacted their progress when they became stuck. For the analysis, we triangulated data across multiple sources (e.g., think-aloud, code changes, web searches, and LLM prompts). Unexpectedly, while developers in the AI group repeatedly turned to the AI tool to offload certain aspects of the process, all detailed problem-solving behaviors appeared in both groups. We also found that nine out of ten participants found themselves stuck in their work, but with key differences in how they became stuck and unstuck. We highlight seven distinct causes for being stuck and highlight how AI in some cases helped and in other cases hindered becoming unstuck. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_10702 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ChatGPT: Friend or Foe When Comprehending and Changing Unfamiliar Code Anderson, Norman Alakmeh, Tarek Jackson, Victoria Pereira, Guilherme Vaz Akirmak, Umit Estey, Anthony Prikladnicki, Rafael Fritz, Thomas van der Hoek, André Storey, Margaret-Anne Software Engineering D.2.9 A rapidly growing body of research is examining how LLMs influence developers when they code. To date, this research has tended to focus on productivity and code quality outcomes, rather than the underlying cognitive processes involved in programming. To address this gap, we report on the results of an exploratory laboratory study of ten advanced student developers (five with support from AI and five without) who had to make a non-trivial extension to a sizable software system. Leveraging Polya's four problem-solving phases and 25 inductively-generated codes detailing distinct problem-solving behaviors as the primary lenses, we examined: (1) how AI impacted the problem-solving approach the developers used to solve the programming task, and (2) how AI impacted their progress when they became stuck. For the analysis, we triangulated data across multiple sources (e.g., think-aloud, code changes, web searches, and LLM prompts). Unexpectedly, while developers in the AI group repeatedly turned to the AI tool to offload certain aspects of the process, all detailed problem-solving behaviors appeared in both groups. We also found that nine out of ten participants found themselves stuck in their work, but with key differences in how they became stuck and unstuck. We highlight seven distinct causes for being stuck and highlight how AI in some cases helped and in other cases hindered becoming unstuck. |
| title | ChatGPT: Friend or Foe When Comprehending and Changing Unfamiliar Code |
| topic | Software Engineering D.2.9 |
| url | https://arxiv.org/abs/2605.10702 |