Investigating the Role of LLMs Hyperparameter Tuning and Prompt Engineering to Support Domain Modeling
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918098714492928 |
|---|---|
| author | Bulhakov, Vladyslav d'Aloisio, Giordano Di Sipio, Claudio Di Marco, Antinisca Di Ruscio, Davide |
| author_facet | Bulhakov, Vladyslav d'Aloisio, Giordano Di Sipio, Claudio Di Marco, Antinisca Di Ruscio, Davide |
| contents | The introduction of large language models (LLMs) has enhanced automation in software engineering tasks, including in Model Driven Engineering (MDE). However, using general-purpose LLMs for domain modeling has its limitations. One approach is to adopt fine-tuned models, but this requires significant computational resources and can lead to issues like catastrophic forgetting.
This paper explores how hyperparameter tuning and prompt engineering can improve the accuracy of the Llama 3.1 model for generating domain models from textual descriptions. We use search-based methods to tune hyperparameters for a specific medical data model, resulting in a notable quality improvement over the baseline LLM. We then test the optimized hyperparameters across ten diverse application domains.
While the solutions were not universally applicable, we demonstrate that combining hyperparameter tuning with prompt engineering can enhance results across nearly all examined domain models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_14735 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Investigating the Role of LLMs Hyperparameter Tuning and Prompt Engineering to Support Domain Modeling Bulhakov, Vladyslav d'Aloisio, Giordano Di Sipio, Claudio Di Marco, Antinisca Di Ruscio, Davide Software Engineering The introduction of large language models (LLMs) has enhanced automation in software engineering tasks, including in Model Driven Engineering (MDE). However, using general-purpose LLMs for domain modeling has its limitations. One approach is to adopt fine-tuned models, but this requires significant computational resources and can lead to issues like catastrophic forgetting. This paper explores how hyperparameter tuning and prompt engineering can improve the accuracy of the Llama 3.1 model for generating domain models from textual descriptions. We use search-based methods to tune hyperparameters for a specific medical data model, resulting in a notable quality improvement over the baseline LLM. We then test the optimized hyperparameters across ten diverse application domains. While the solutions were not universally applicable, we demonstrate that combining hyperparameter tuning with prompt engineering can enhance results across nearly all examined domain models. |
| title | Investigating the Role of LLMs Hyperparameter Tuning and Prompt Engineering to Support Domain Modeling |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2507.14735 |