Introducing MAPO: Momentum-Aided Gradient Descent Prompt Optimization
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908424549171200 |
|---|---|
| author | Cui, Anthony Nandyalam, Pranav Rufail, Andrew Cheung, Ethan Lei, Aiden Zhu, Kevin O'Brien, Sean |
| author_facet | Cui, Anthony Nandyalam, Pranav Rufail, Andrew Cheung, Ethan Lei, Aiden Zhu, Kevin O'Brien, Sean |
| contents | Momentum-Aided Prompt Optimization (MAPO) enhances the efficiency and efficacy of prompt optimization for Large Language Models (LLMs). Building on ProTeGi, MAPO uses positive natural language "gradients" and a momentum-based extension to refine prompts effectively. By tracking gradient history, MAPO avoids local minima and oscillations. It also utilizes beam search and an Upper Confidence Bound (UCB) algorithm for balanced candidate expansion and selection. Benchmark testing shows that MAPO achieves faster convergence time with fewer API calls and higher F1 scores than ProTeGi, proving it as a robust and scalable solution for automated prompt engineering in LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_19499 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Introducing MAPO: Momentum-Aided Gradient Descent Prompt Optimization Cui, Anthony Nandyalam, Pranav Rufail, Andrew Cheung, Ethan Lei, Aiden Zhu, Kevin O'Brien, Sean Computation and Language Momentum-Aided Prompt Optimization (MAPO) enhances the efficiency and efficacy of prompt optimization for Large Language Models (LLMs). Building on ProTeGi, MAPO uses positive natural language "gradients" and a momentum-based extension to refine prompts effectively. By tracking gradient history, MAPO avoids local minima and oscillations. It also utilizes beam search and an Upper Confidence Bound (UCB) algorithm for balanced candidate expansion and selection. Benchmark testing shows that MAPO achieves faster convergence time with fewer API calls and higher F1 scores than ProTeGi, proving it as a robust and scalable solution for automated prompt engineering in LLMs. |
| title | Introducing MAPO: Momentum-Aided Gradient Descent Prompt Optimization |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2410.19499 |