Prompting for Performance: Exploring LLMs for Configuring Software
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912600616337408 |
|---|---|
| author | Spieker, Helge Matricon, Théo Belmecheri, Nassim Betten, Jørn Eirik Lyan, Gauthier Le Bartz Borges, Heraldo Mazouni, Quentin Gross, Dennis Gotlieb, Arnaud Acher, Mathieu |
| author_facet | Spieker, Helge Matricon, Théo Belmecheri, Nassim Betten, Jørn Eirik Lyan, Gauthier Le Bartz Borges, Heraldo Mazouni, Quentin Gross, Dennis Gotlieb, Arnaud Acher, Mathieu |
| contents | Software systems usually provide numerous configuration options that can affect performance metrics such as execution time, memory usage, binary size, or bitrate. On the one hand, making informed decisions is challenging and requires domain expertise in options and their combinations. On the other hand, machine learning techniques can search vast configuration spaces, but with a high computational cost, since concrete executions of numerous configurations are required. In this exploratory study, we investigate whether large language models (LLMs) can assist in performance-oriented software configuration through prompts. We evaluate several LLMs on tasks including identifying relevant options, ranking configurations, and recommending performant configurations across various configurable systems, such as compilers, video encoders, and SAT solvers. Our preliminary results reveal both positive abilities and notable limitations: depending on the task and systems, LLMs can well align with expert knowledge, whereas hallucinations or superficial reasoning can emerge in other cases. These findings represent a first step toward systematic evaluations and the design of LLM-based solutions to assist with software configuration. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_09790 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Prompting for Performance: Exploring LLMs for Configuring Software Spieker, Helge Matricon, Théo Belmecheri, Nassim Betten, Jørn Eirik Lyan, Gauthier Le Bartz Borges, Heraldo Mazouni, Quentin Gross, Dennis Gotlieb, Arnaud Acher, Mathieu Software Engineering Artificial Intelligence Performance Software systems usually provide numerous configuration options that can affect performance metrics such as execution time, memory usage, binary size, or bitrate. On the one hand, making informed decisions is challenging and requires domain expertise in options and their combinations. On the other hand, machine learning techniques can search vast configuration spaces, but with a high computational cost, since concrete executions of numerous configurations are required. In this exploratory study, we investigate whether large language models (LLMs) can assist in performance-oriented software configuration through prompts. We evaluate several LLMs on tasks including identifying relevant options, ranking configurations, and recommending performant configurations across various configurable systems, such as compilers, video encoders, and SAT solvers. Our preliminary results reveal both positive abilities and notable limitations: depending on the task and systems, LLMs can well align with expert knowledge, whereas hallucinations or superficial reasoning can emerge in other cases. These findings represent a first step toward systematic evaluations and the design of LLM-based solutions to assist with software configuration. |
| title | Prompting for Performance: Exploring LLMs for Configuring Software |
| topic | Software Engineering Artificial Intelligence Performance |
| url | https://arxiv.org/abs/2507.09790 |