How Do Agents Perform Code Optimization? An Empirical Study
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914220557205504 |
|---|---|
| author | Peng, Huiyun Zhong, Antonio Méndez, Ricardo Andrés Calvo Kalu, Kelechi G. Davis, James C. |
| author_facet | Peng, Huiyun Zhong, Antonio Méndez, Ricardo Andrés Calvo Kalu, Kelechi G. Davis, James C. |
| contents | Performance optimization is a critical yet challenging aspect of software development, often requiring a deep understanding of system behavior, algorithmic tradeoffs, and careful code modifications. Although recent advances in AI coding agents have accelerated code generation and bug fixing, little is known about how these agents perform on real-world performance optimization tasks. We present the first empirical study comparing agent- and human-authored performance optimization commits, analyzing 324 agent-generated and 83 human-authored PRs from the AIDev dataset across adoption, maintainability, optimization patterns, and validation practices. We find that AI-authored performance PRs are less likely to include explicit performance validation than human-authored PRs (45.7\% vs. 63.6\%, $p=0.007$). In addition, AI-authored PRs largely use the same optimization patterns as humans. We further discuss limitations and opportunities for advancing agentic code optimization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_21757 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | How Do Agents Perform Code Optimization? An Empirical Study Peng, Huiyun Zhong, Antonio Méndez, Ricardo Andrés Calvo Kalu, Kelechi G. Davis, James C. Software Engineering Artificial Intelligence Performance optimization is a critical yet challenging aspect of software development, often requiring a deep understanding of system behavior, algorithmic tradeoffs, and careful code modifications. Although recent advances in AI coding agents have accelerated code generation and bug fixing, little is known about how these agents perform on real-world performance optimization tasks. We present the first empirical study comparing agent- and human-authored performance optimization commits, analyzing 324 agent-generated and 83 human-authored PRs from the AIDev dataset across adoption, maintainability, optimization patterns, and validation practices. We find that AI-authored performance PRs are less likely to include explicit performance validation than human-authored PRs (45.7\% vs. 63.6\%, $p=0.007$). In addition, AI-authored PRs largely use the same optimization patterns as humans. We further discuss limitations and opportunities for advancing agentic code optimization. |
| title | How Do Agents Perform Code Optimization? An Empirical Study |
| topic | Software Engineering Artificial Intelligence |
| url | https://arxiv.org/abs/2512.21757 |