Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.15669 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913747248873472 |
|---|---|
| author | Lin, Hannah Maas, Martin Roquemore, Maximilian Hasanzadeh, Arman Lewis, Fred Simonson, Yusuf Yang, Tzu-Wei Yazdanbakhsh, Amir Altinbüken, Deniz Papa, Florin Edmonds, Maggie Nolan Patil, Aditya Schwarz, Don Chandra, Satish Kennelly, Chris Hashemi, Milad Ranganathan, Parthasarathy |
| author_facet | Lin, Hannah Maas, Martin Roquemore, Maximilian Hasanzadeh, Arman Lewis, Fred Simonson, Yusuf Yang, Tzu-Wei Yazdanbakhsh, Amir Altinbüken, Deniz Papa, Florin Edmonds, Maggie Nolan Patil, Aditya Schwarz, Don Chandra, Satish Kennelly, Chris Hashemi, Milad Ranganathan, Parthasarathy |
| contents | With the end of Moore's Law, optimizing code for performance has become paramount for meeting ever-increasing compute demands, particularly in hyperscale data centers where even small efficiency gains translate to significant resource and energy savings. Traditionally, this process requires significant programmer effort to identify optimization opportunities, modify the code to implement the optimization, and carefully deploy and measure the optimization's impact. Despite a significant amount of work on automating program edits and promising results in small-scale settings, such performance optimizations have remained elusive in large real-world production environments, due to the scale, high degree of complexity, and reliability required.
This paper introduces ECO (Efficient Code Optimizer), a system that automatically refactors source code to improve performance at scale. To achieve these performance gains, ECO searches through historical commits at scale to create a dictionary of performance anti-patterns that these commits addressed. These anti-patterns are used to search for similar patterns in a code base of billions of lines of code, pinpointing other code segments with similar potential optimization opportunities. Using a fine-tuned LLM, ECO then automatically refactors the code to generate and apply similar edits. Next, ECO verifies the transformed code, submits it for code review, and measures the impact of the optimization in production.
Currently deployed on Google's hyperscale production fleet, this system has driven >25k changed lines of production code, across over 6.4k submitted commits, with a >99.5% production success rate. Over the past year, ECO has consistently resulted in significant performance savings every quarter. On average, the savings produced per quarter are equivalent to over 500k normalized CPU cores. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_15669 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ECO: An LLM-Driven Efficient Code Optimizer for Warehouse Scale Computers Lin, Hannah Maas, Martin Roquemore, Maximilian Hasanzadeh, Arman Lewis, Fred Simonson, Yusuf Yang, Tzu-Wei Yazdanbakhsh, Amir Altinbüken, Deniz Papa, Florin Edmonds, Maggie Nolan Patil, Aditya Schwarz, Don Chandra, Satish Kennelly, Chris Hashemi, Milad Ranganathan, Parthasarathy Software Engineering With the end of Moore's Law, optimizing code for performance has become paramount for meeting ever-increasing compute demands, particularly in hyperscale data centers where even small efficiency gains translate to significant resource and energy savings. Traditionally, this process requires significant programmer effort to identify optimization opportunities, modify the code to implement the optimization, and carefully deploy and measure the optimization's impact. Despite a significant amount of work on automating program edits and promising results in small-scale settings, such performance optimizations have remained elusive in large real-world production environments, due to the scale, high degree of complexity, and reliability required. This paper introduces ECO (Efficient Code Optimizer), a system that automatically refactors source code to improve performance at scale. To achieve these performance gains, ECO searches through historical commits at scale to create a dictionary of performance anti-patterns that these commits addressed. These anti-patterns are used to search for similar patterns in a code base of billions of lines of code, pinpointing other code segments with similar potential optimization opportunities. Using a fine-tuned LLM, ECO then automatically refactors the code to generate and apply similar edits. Next, ECO verifies the transformed code, submits it for code review, and measures the impact of the optimization in production. Currently deployed on Google's hyperscale production fleet, this system has driven >25k changed lines of production code, across over 6.4k submitted commits, with a >99.5% production success rate. Over the past year, ECO has consistently resulted in significant performance savings every quarter. On average, the savings produced per quarter are equivalent to over 500k normalized CPU cores. |
| title | ECO: An LLM-Driven Efficient Code Optimizer for Warehouse Scale Computers |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2503.15669 |