Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2511.15915 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915939089383424 |
|---|---|
| author | Zhang, Genghan Zhu, Shaowei Wei, Anjiang Song, Zhenyu Nie, Allen Jia, Zhen Vijaykumar, Nandita Wang, Yida Olukotun, Kunle |
| author_facet | Zhang, Genghan Zhu, Shaowei Wei, Anjiang Song, Zhenyu Nie, Allen Jia, Zhen Vijaykumar, Nandita Wang, Yida Olukotun, Kunle |
| contents | We present AccelOpt, a self-improving large language model (LLM) agentic system that autonomously optimizes kernels for emerging AI acclerators, eliminating the need for expert-provided hardware-specific optimization knowledge. AccelOpt explores the kernel optimization space through iterative generation, informed by an optimization memory that curates experiences and insights from previously encountered slow-fast kernel pairs. We build NKIBench, a new benchmark suite of AWS Trainium accelerator kernels with varying complexity extracted from real-world LLM workloads to evaluate the effectiveness of AccelOpt. Our evaluation confirms that AccelOpt's capability improves over time, boosting the average percentage of peak throughput from $49\%$ to $61\%$ on Trainium 1 and from $45\%$ to $59\%$ on Trainium 2 for NKIBench kernels. Moreover, AccelOpt is highly cost-effective: using open-source models, it matches the kernel improvements of Claude Sonnet 4 while being $26\times$ cheaper. The code is open-sourced at https://github.com/zhang677/AccelOpt. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_15915 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AccelOpt: A Self-Improving LLM Agentic System for AI Accelerator Kernel Optimization Zhang, Genghan Zhu, Shaowei Wei, Anjiang Song, Zhenyu Nie, Allen Jia, Zhen Vijaykumar, Nandita Wang, Yida Olukotun, Kunle Machine Learning Computation and Language We present AccelOpt, a self-improving large language model (LLM) agentic system that autonomously optimizes kernels for emerging AI acclerators, eliminating the need for expert-provided hardware-specific optimization knowledge. AccelOpt explores the kernel optimization space through iterative generation, informed by an optimization memory that curates experiences and insights from previously encountered slow-fast kernel pairs. We build NKIBench, a new benchmark suite of AWS Trainium accelerator kernels with varying complexity extracted from real-world LLM workloads to evaluate the effectiveness of AccelOpt. Our evaluation confirms that AccelOpt's capability improves over time, boosting the average percentage of peak throughput from $49\%$ to $61\%$ on Trainium 1 and from $45\%$ to $59\%$ on Trainium 2 for NKIBench kernels. Moreover, AccelOpt is highly cost-effective: using open-source models, it matches the kernel improvements of Claude Sonnet 4 while being $26\times$ cheaper. The code is open-sourced at https://github.com/zhang677/AccelOpt. |
| title | AccelOpt: A Self-Improving LLM Agentic System for AI Accelerator Kernel Optimization |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2511.15915 |