LLMPerf: GPU Performance Modeling meets Large Language Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929759844302848 |
|---|---|
| author | Nguyen, Khoi N. M. Do, Hoang Duy Nguyen Le, Huyen Thao Dao, Thanh Tuan |
| author_facet | Nguyen, Khoi N. M. Do, Hoang Duy Nguyen Le, Huyen Thao Dao, Thanh Tuan |
| contents | Performance modeling, a pivotal domain in program cost analysis, currently relies on manually crafted models constrained by various program and hardware limitations, especially in the intricate landscape of GPGPU. Meanwhile, Large Language Models (LLMs) have demonstrated their effectiveness in addressing diverse programming challenges. Our work establishes a connection between LLMs and performance modeling, employing the LLM as a performance estimator. Through experimental exploration with carefully designed large-scale OpenCL datasets, we highlight the potential capability as well as the main difficulties of using LLMs in handling performance modeling tasks for OpenCL device source programs. As the first study for this line of work, our LLM-based performance model achieves a mean absolute percentage error of $24.25\%$ for a large-scale generated validation set. On a set of publicly available OpenCL programs, our model achieves a mean absolute percentage error of $46.1\%$. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_11244 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LLMPerf: GPU Performance Modeling meets Large Language Models Nguyen, Khoi N. M. Do, Hoang Duy Nguyen Le, Huyen Thao Dao, Thanh Tuan Performance Distributed, Parallel, and Cluster Computing Machine Learning Performance modeling, a pivotal domain in program cost analysis, currently relies on manually crafted models constrained by various program and hardware limitations, especially in the intricate landscape of GPGPU. Meanwhile, Large Language Models (LLMs) have demonstrated their effectiveness in addressing diverse programming challenges. Our work establishes a connection between LLMs and performance modeling, employing the LLM as a performance estimator. Through experimental exploration with carefully designed large-scale OpenCL datasets, we highlight the potential capability as well as the main difficulties of using LLMs in handling performance modeling tasks for OpenCL device source programs. As the first study for this line of work, our LLM-based performance model achieves a mean absolute percentage error of $24.25\%$ for a large-scale generated validation set. On a set of publicly available OpenCL programs, our model achieves a mean absolute percentage error of $46.1\%$. |
| title | LLMPerf: GPU Performance Modeling meets Large Language Models |
| topic | Performance Distributed, Parallel, and Cluster Computing Machine Learning |
| url | https://arxiv.org/abs/2503.11244 |