Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2508.01443 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908574119100416 |
|---|---|
| author | Gong, Jingzhi Giavrimis, Rafail Brookes, Paul Voskanyan, Vardan Wu, Fan Ashiga, Mari Truscott, Matthew Basios, Mike Kanthan, Leslie Xu, Jie Wang, Zheng |
| author_facet | Gong, Jingzhi Giavrimis, Rafail Brookes, Paul Voskanyan, Vardan Wu, Fan Ashiga, Mari Truscott, Matthew Basios, Mike Kanthan, Leslie Xu, Jie Wang, Zheng |
| contents | There is a growing interest in leveraging multiple large language models (LLMs) for automated code optimization. However, industrial platforms deploying multiple LLMs face a critical challenge: prompts optimized for one LLM often fail with others, requiring expensive model-specific prompt engineering. This cross-model prompt engineering bottleneck severely limits the practical deployment of multi-LLM systems in production environments. We introduce Meta-Prompted Code Optimization (MPCO), a framework that automatically generates high-quality, task-specific prompts across diverse LLMs while maintaining industrial efficiency requirements. MPCO leverages metaprompting to dynamically synthesize context-aware optimization prompts by integrating project metadata, task requirements, and LLM-specific contexts. It is an essential part of the ARTEMIS code optimization platform for automated validation and scaling. Our comprehensive evaluation on five real-world codebases with 366 hours of runtime benchmarking demonstrates MPCO's effectiveness: it achieves overall performance improvements up to 19.06% with the best statistical rank across all systems compared to baseline methods. Analysis shows that 96% of the top-performing optimizations stem from meaningful edits. Through systematic ablation studies and meta-prompter sensitivity analysis, we identify that comprehensive context integration is essential for effective meta-prompting and that major LLMs can serve effectively as meta-prompters, providing actionable insights for industrial practitioners. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_01443 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Tuning LLM-based Code Optimization via Meta-Prompting: An Industrial Perspective Gong, Jingzhi Giavrimis, Rafail Brookes, Paul Voskanyan, Vardan Wu, Fan Ashiga, Mari Truscott, Matthew Basios, Mike Kanthan, Leslie Xu, Jie Wang, Zheng Software Engineering Artificial Intelligence There is a growing interest in leveraging multiple large language models (LLMs) for automated code optimization. However, industrial platforms deploying multiple LLMs face a critical challenge: prompts optimized for one LLM often fail with others, requiring expensive model-specific prompt engineering. This cross-model prompt engineering bottleneck severely limits the practical deployment of multi-LLM systems in production environments. We introduce Meta-Prompted Code Optimization (MPCO), a framework that automatically generates high-quality, task-specific prompts across diverse LLMs while maintaining industrial efficiency requirements. MPCO leverages metaprompting to dynamically synthesize context-aware optimization prompts by integrating project metadata, task requirements, and LLM-specific contexts. It is an essential part of the ARTEMIS code optimization platform for automated validation and scaling. Our comprehensive evaluation on five real-world codebases with 366 hours of runtime benchmarking demonstrates MPCO's effectiveness: it achieves overall performance improvements up to 19.06% with the best statistical rank across all systems compared to baseline methods. Analysis shows that 96% of the top-performing optimizations stem from meaningful edits. Through systematic ablation studies and meta-prompter sensitivity analysis, we identify that comprehensive context integration is essential for effective meta-prompting and that major LLMs can serve effectively as meta-prompters, providing actionable insights for industrial practitioners. |
| title | Tuning LLM-based Code Optimization via Meta-Prompting: An Industrial Perspective |
| topic | Software Engineering Artificial Intelligence |
| url | https://arxiv.org/abs/2508.01443 |