Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2407.12865 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929425329684480 |
|---|---|
| author | Austin, Derek Chartock, Elliott |
| author_facet | Austin, Derek Chartock, Elliott |
| contents | Prompt engineering for large language models (LLMs) is often a manual time-intensive process that involves generating, evaluating, and refining prompts iteratively to ensure high-quality outputs. While there has been work on automating prompt engineering, the solutions generally are either tuned to specific tasks with given answers or are quite costly. We introduce GRAD-SUM, a scalable and flexible method for automatic prompt engineering that builds on gradient-based optimization techniques. Our approach incorporates user-defined task descriptions and evaluation criteria, and features a novel gradient summarization module to generalize feedback effectively. Our results demonstrate that GRAD-SUM consistently outperforms existing methods across various benchmarks, highlighting its versatility and effectiveness in automatic prompt optimization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_12865 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | GRAD-SUM: Leveraging Gradient Summarization for Optimal Prompt Engineering Austin, Derek Chartock, Elliott Computation and Language Artificial Intelligence Prompt engineering for large language models (LLMs) is often a manual time-intensive process that involves generating, evaluating, and refining prompts iteratively to ensure high-quality outputs. While there has been work on automating prompt engineering, the solutions generally are either tuned to specific tasks with given answers or are quite costly. We introduce GRAD-SUM, a scalable and flexible method for automatic prompt engineering that builds on gradient-based optimization techniques. Our approach incorporates user-defined task descriptions and evaluation criteria, and features a novel gradient summarization module to generalize feedback effectively. Our results demonstrate that GRAD-SUM consistently outperforms existing methods across various benchmarks, highlighting its versatility and effectiveness in automatic prompt optimization. |
| title | GRAD-SUM: Leveraging Gradient Summarization for Optimal Prompt Engineering |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2407.12865 |