Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.16465 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908729679544320 |
|---|---|
| author | Chen, Jinwu Wu, Qidie Li, Bin Ma, Lin Si, Xin Hu, Yang Yin, Shouyi Yang, Jun |
| author_facet | Chen, Jinwu Wu, Qidie Li, Bin Ma, Lin Si, Xin Hu, Yang Yin, Shouyi Yang, Jun |
| contents | Optimizing CUDA kernels is a challenging and labor-intensive task, given the need for hardware-software co-design expertise and the proprietary nature of high-performance kernel libraries. While recent large language models (LLMs) combined with evolutionary algorithms show promise in automatic kernel optimization, existing approaches often fall short in performance due to their suboptimal agent designs and mismatched evolution representations. This work identifies these mismatches and proposes cuPilot, a strategy-coordinated multi-agent framework that introduces strategy as an intermediate semantic representation for kernel evolution. Key contributions include a strategy-coordinated evolution algorithm, roofline-guided prompting, and strategy-level population initialization. Experimental results show that the generated kernels by cuPilot achieve an average speed up of 3.09$\times$ over PyTorch on a benchmark of 100 kernels. On the GEMM tasks, cuPilot showcases sophisticated optimizations and achieves high utilization of critical hardware units. The generated kernels are open-sourced at https://github.com/champloo2878/cuPilot-Kernels.git. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_16465 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | cuPilot: A Strategy-Coordinated Multi-agent Framework for CUDA Kernel Evolution Chen, Jinwu Wu, Qidie Li, Bin Ma, Lin Si, Xin Hu, Yang Yin, Shouyi Yang, Jun Artificial Intelligence Optimizing CUDA kernels is a challenging and labor-intensive task, given the need for hardware-software co-design expertise and the proprietary nature of high-performance kernel libraries. While recent large language models (LLMs) combined with evolutionary algorithms show promise in automatic kernel optimization, existing approaches often fall short in performance due to their suboptimal agent designs and mismatched evolution representations. This work identifies these mismatches and proposes cuPilot, a strategy-coordinated multi-agent framework that introduces strategy as an intermediate semantic representation for kernel evolution. Key contributions include a strategy-coordinated evolution algorithm, roofline-guided prompting, and strategy-level population initialization. Experimental results show that the generated kernels by cuPilot achieve an average speed up of 3.09$\times$ over PyTorch on a benchmark of 100 kernels. On the GEMM tasks, cuPilot showcases sophisticated optimizations and achieves high utilization of critical hardware units. The generated kernels are open-sourced at https://github.com/champloo2878/cuPilot-Kernels.git. |
| title | cuPilot: A Strategy-Coordinated Multi-agent Framework for CUDA Kernel Evolution |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2512.16465 |