SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909385046884352 |
|---|---|
| author | Yang, John Jimenez, Carlos E. Wettig, Alexander Lieret, Kilian Yao, Shunyu Narasimhan, Karthik Press, Ofir |
| author_facet | Yang, John Jimenez, Carlos E. Wettig, Alexander Lieret, Kilian Yao, Shunyu Narasimhan, Karthik Press, Ofir |
| contents | Language model (LM) agents are increasingly being used to automate complicated tasks in digital environments. Just as humans benefit from powerful software applications, such as integrated development environments, for complex tasks like software engineering, we posit that LM agents represent a new category of end users with their own needs and abilities, and would benefit from specially-built interfaces to the software they use. We investigate how interface design affects the performance of language model agents. As a result of this exploration, we introduce SWE-agent: a system that facilitates LM agents to autonomously use computers to solve software engineering tasks. SWE-agent's custom agent-computer interface (ACI) significantly enhances an agent's ability to create and edit code files, navigate entire repositories, and execute tests and other programs. We evaluate SWE-agent on SWE-bench and HumanEvalFix, achieving state-of-the-art performance on both with a pass@1 rate of 12.5% and 87.7%, respectively, far exceeding the previous state-of-the-art achieved with non-interactive LMs. Finally, we provide insight on how the design of the ACI can impact agents' behavior and performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_15793 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering Yang, John Jimenez, Carlos E. Wettig, Alexander Lieret, Kilian Yao, Shunyu Narasimhan, Karthik Press, Ofir Software Engineering Artificial Intelligence Computation and Language Human-Computer Interaction Machine Learning Language model (LM) agents are increasingly being used to automate complicated tasks in digital environments. Just as humans benefit from powerful software applications, such as integrated development environments, for complex tasks like software engineering, we posit that LM agents represent a new category of end users with their own needs and abilities, and would benefit from specially-built interfaces to the software they use. We investigate how interface design affects the performance of language model agents. As a result of this exploration, we introduce SWE-agent: a system that facilitates LM agents to autonomously use computers to solve software engineering tasks. SWE-agent's custom agent-computer interface (ACI) significantly enhances an agent's ability to create and edit code files, navigate entire repositories, and execute tests and other programs. We evaluate SWE-agent on SWE-bench and HumanEvalFix, achieving state-of-the-art performance on both with a pass@1 rate of 12.5% and 87.7%, respectively, far exceeding the previous state-of-the-art achieved with non-interactive LMs. Finally, we provide insight on how the design of the ACI can impact agents' behavior and performance. |
| title | SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering |
| topic | Software Engineering Artificial Intelligence Computation and Language Human-Computer Interaction Machine Learning |
| url | https://arxiv.org/abs/2405.15793 |