HPC-Coder-V2: Studying Code LLMs Across Low-Resource Parallel Languages
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913620027244544 |
|---|---|
| author | Chaturvedi, Aman Nichols, Daniel Singh, Siddharth Bhatele, Abhinav |
| author_facet | Chaturvedi, Aman Nichols, Daniel Singh, Siddharth Bhatele, Abhinav |
| contents | Large Language Model (LLM) based coding tools have been tremendously successful as software development assistants, yet they are often designed for general purpose programming tasks and perform poorly for more specialized domains such as high performance computing. Creating specialized models and tools for these domains is crucial towards gaining the benefits of LLMs in areas such as HPC. While previous work has explored HPC-specific models, LLMs still struggle to generate parallel code and it is not at all clear what hurdles are still holding back these LLMs and what must be done to overcome them. In this work, we conduct an in-depth study along the many axes of fine-tuning a specialized HPC LLM in order to better understand the challenges. Based on our findings we fine-tune and evaluate a specialized HPC LLM that is shown to be the best performing open-source code LLM for parallel code generation to date. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_15178 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | HPC-Coder-V2: Studying Code LLMs Across Low-Resource Parallel Languages Chaturvedi, Aman Nichols, Daniel Singh, Siddharth Bhatele, Abhinav Distributed, Parallel, and Cluster Computing Machine Learning Software Engineering Large Language Model (LLM) based coding tools have been tremendously successful as software development assistants, yet they are often designed for general purpose programming tasks and perform poorly for more specialized domains such as high performance computing. Creating specialized models and tools for these domains is crucial towards gaining the benefits of LLMs in areas such as HPC. While previous work has explored HPC-specific models, LLMs still struggle to generate parallel code and it is not at all clear what hurdles are still holding back these LLMs and what must be done to overcome them. In this work, we conduct an in-depth study along the many axes of fine-tuning a specialized HPC LLM in order to better understand the challenges. Based on our findings we fine-tune and evaluate a specialized HPC LLM that is shown to be the best performing open-source code LLM for parallel code generation to date. |
| title | HPC-Coder-V2: Studying Code LLMs Across Low-Resource Parallel Languages |
| topic | Distributed, Parallel, and Cluster Computing Machine Learning Software Engineering |
| url | https://arxiv.org/abs/2412.15178 |