OMPGPT: A Generative Pre-trained Transformer Model for OpenMP
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916470580051968 |
|---|---|
| author | Chen, Le Bhattacharjee, Arijit Ahmed, Nesreen Hasabnis, Niranjan Oren, Gal Vo, Vy Jannesari, Ali |
| author_facet | Chen, Le Bhattacharjee, Arijit Ahmed, Nesreen Hasabnis, Niranjan Oren, Gal Vo, Vy Jannesari, Ali |
| contents | Large language models (LLMs)such as ChatGPT have significantly advanced the field of Natural Language Processing (NLP). This trend led to the development of code-based large language models such as StarCoder, WizardCoder, and CodeLlama, which are trained extensively on vast repositories of code and programming languages. While the generic abilities of these code LLMs are useful for many programmers in tasks like code generation, the area of high-performance computing (HPC) has a narrower set of requirements that make a smaller and more domain-specific model a smarter choice. This paper presents OMPGPT, a novel domain-specific model meticulously designed to harness the inherent strengths of language models for OpenMP pragma generation. Furthermore, we leverage prompt engineering techniques from the NLP domain to create Chain-of-OMP, an innovative strategy designed to enhance OMPGPT's effectiveness. Our extensive evaluations demonstrate that OMPGPT outperforms existing large language models specialized in OpenMP tasks and maintains a notably smaller size, aligning it more closely with the typical hardware constraints of HPC environments. We consider our contribution as a pivotal bridge, connecting the advantage of language models with the specific demands of HPC tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_16445 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | OMPGPT: A Generative Pre-trained Transformer Model for OpenMP Chen, Le Bhattacharjee, Arijit Ahmed, Nesreen Hasabnis, Niranjan Oren, Gal Vo, Vy Jannesari, Ali Software Engineering Distributed, Parallel, and Cluster Computing Machine Learning Large language models (LLMs)such as ChatGPT have significantly advanced the field of Natural Language Processing (NLP). This trend led to the development of code-based large language models such as StarCoder, WizardCoder, and CodeLlama, which are trained extensively on vast repositories of code and programming languages. While the generic abilities of these code LLMs are useful for many programmers in tasks like code generation, the area of high-performance computing (HPC) has a narrower set of requirements that make a smaller and more domain-specific model a smarter choice. This paper presents OMPGPT, a novel domain-specific model meticulously designed to harness the inherent strengths of language models for OpenMP pragma generation. Furthermore, we leverage prompt engineering techniques from the NLP domain to create Chain-of-OMP, an innovative strategy designed to enhance OMPGPT's effectiveness. Our extensive evaluations demonstrate that OMPGPT outperforms existing large language models specialized in OpenMP tasks and maintains a notably smaller size, aligning it more closely with the typical hardware constraints of HPC environments. We consider our contribution as a pivotal bridge, connecting the advantage of language models with the specific demands of HPC tasks. |
| title | OMPGPT: A Generative Pre-trained Transformer Model for OpenMP |
| topic | Software Engineering Distributed, Parallel, and Cluster Computing Machine Learning |
| url | https://arxiv.org/abs/2401.16445 |