PromptPilot: Improving Human-AI Collaboration Through LLM-Enhanced Prompt Engineering
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909818336313344 |
|---|---|
| author | Gutheil, Niklas Mayer, Valentin Müller, Leopold Rommelt, Jörg Kühl, Niklas |
| author_facet | Gutheil, Niklas Mayer, Valentin Müller, Leopold Rommelt, Jörg Kühl, Niklas |
| contents | Effective prompt engineering is critical to realizing the promised productivity gains of large language models (LLMs) in knowledge-intensive tasks. Yet, many users struggle to craft prompts that yield high-quality outputs, limiting the practical benefits of LLMs. Existing approaches, such as prompt handbooks or automated optimization pipelines, either require substantial effort, expert knowledge, or lack interactive guidance. To address this gap, we design and evaluate PromptPilot, an interactive prompting assistant grounded in four empirically derived design objectives for LLM-enhanced prompt engineering. We conducted a randomized controlled experiment with 80 participants completing three realistic, work-related writing tasks. Participants supported by PromptPilot achieved significantly higher performance (median: 78.3 vs. 61.7; p = .045, d = 0.56), and reported enhanced efficiency, ease-of-use, and autonomy during interaction. These findings empirically validate the effectiveness of our proposed design objectives, establishing LLM-enhanced prompt engineering as a viable technique for improving human-AI collaboration. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_00555 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PromptPilot: Improving Human-AI Collaboration Through LLM-Enhanced Prompt Engineering Gutheil, Niklas Mayer, Valentin Müller, Leopold Rommelt, Jörg Kühl, Niklas Human-Computer Interaction Artificial Intelligence Effective prompt engineering is critical to realizing the promised productivity gains of large language models (LLMs) in knowledge-intensive tasks. Yet, many users struggle to craft prompts that yield high-quality outputs, limiting the practical benefits of LLMs. Existing approaches, such as prompt handbooks or automated optimization pipelines, either require substantial effort, expert knowledge, or lack interactive guidance. To address this gap, we design and evaluate PromptPilot, an interactive prompting assistant grounded in four empirically derived design objectives for LLM-enhanced prompt engineering. We conducted a randomized controlled experiment with 80 participants completing three realistic, work-related writing tasks. Participants supported by PromptPilot achieved significantly higher performance (median: 78.3 vs. 61.7; p = .045, d = 0.56), and reported enhanced efficiency, ease-of-use, and autonomy during interaction. These findings empirically validate the effectiveness of our proposed design objectives, establishing LLM-enhanced prompt engineering as a viable technique for improving human-AI collaboration. |
| title | PromptPilot: Improving Human-AI Collaboration Through LLM-Enhanced Prompt Engineering |
| topic | Human-Computer Interaction Artificial Intelligence |
| url | https://arxiv.org/abs/2510.00555 |