Assistance or Disruption? Exploring and Evaluating the Design and Trade-offs of Proactive AI Programming Support
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918137063014400 |
|---|---|
| author | Pu, Kevin Lazaro, Daniel Arawjo, Ian Xia, Haijun Xiao, Ziang Grossman, Tovi Chen, Yan |
| author_facet | Pu, Kevin Lazaro, Daniel Arawjo, Ian Xia, Haijun Xiao, Ziang Grossman, Tovi Chen, Yan |
| contents | AI programming tools enable powerful code generation, and recent prototypes attempt to reduce user effort with proactive AI agents, but their impact on programming workflows remains unexplored. We introduce and evaluate Codellaborator, a design probe LLM agent that initiates programming assistance based on editor activities and task context. We explored three interface variants to assess trade-offs between increasingly salient AI support: prompt-only, proactive agent, and proactive agent with presence and context (Codellaborator). In a within-subject study (N=18), we find that proactive agents increase efficiency compared to prompt-only paradigm, but also incur workflow disruptions. However, presence indicators and interaction context support alleviated disruptions and improved users' awareness of AI processes. We underscore trade-offs of Codellaborator on user control, ownership, and code understanding, emphasizing the need to adapt proactivity to programming processes. Our research contributes to the design exploration and evaluation of proactive AI systems, presenting design implications on AI-integrated programming workflow. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_18658 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Assistance or Disruption? Exploring and Evaluating the Design and Trade-offs of Proactive AI Programming Support Pu, Kevin Lazaro, Daniel Arawjo, Ian Xia, Haijun Xiao, Ziang Grossman, Tovi Chen, Yan Human-Computer Interaction Artificial Intelligence Software Engineering AI programming tools enable powerful code generation, and recent prototypes attempt to reduce user effort with proactive AI agents, but their impact on programming workflows remains unexplored. We introduce and evaluate Codellaborator, a design probe LLM agent that initiates programming assistance based on editor activities and task context. We explored three interface variants to assess trade-offs between increasingly salient AI support: prompt-only, proactive agent, and proactive agent with presence and context (Codellaborator). In a within-subject study (N=18), we find that proactive agents increase efficiency compared to prompt-only paradigm, but also incur workflow disruptions. However, presence indicators and interaction context support alleviated disruptions and improved users' awareness of AI processes. We underscore trade-offs of Codellaborator on user control, ownership, and code understanding, emphasizing the need to adapt proactivity to programming processes. Our research contributes to the design exploration and evaluation of proactive AI systems, presenting design implications on AI-integrated programming workflow. |
| title | Assistance or Disruption? Exploring and Evaluating the Design and Trade-offs of Proactive AI Programming Support |
| topic | Human-Computer Interaction Artificial Intelligence Software Engineering |
| url | https://arxiv.org/abs/2502.18658 |