The Importance of Directional Feedback for LLM-based Optimizers
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917699281485824 |
|---|---|
| author | Nie, Allen Cheng, Ching-An Kolobov, Andrey Swaminathan, Adith |
| author_facet | Nie, Allen Cheng, Ching-An Kolobov, Andrey Swaminathan, Adith |
| contents | We study the potential of using large language models (LLMs) as an interactive optimizer for solving maximization problems in a text space using natural language and numerical feedback. Inspired by the classical optimization literature, we classify the natural language feedback into directional and non-directional, where the former is a generalization of the first-order feedback to the natural language space. We find that LLMs are especially capable of optimization when they are provided with {directional feedback}. Based on this insight, we design a new LLM-based optimizer that synthesizes directional feedback from the historical optimization trace to achieve reliable improvement over iterations. Empirically, we show our LLM-based optimizer is more stable and efficient in solving optimization problems, from maximizing mathematical functions to optimizing prompts for writing poems, compared with existing techniques. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_16434 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Importance of Directional Feedback for LLM-based Optimizers Nie, Allen Cheng, Ching-An Kolobov, Andrey Swaminathan, Adith Artificial Intelligence Computation and Language Neural and Evolutionary Computing We study the potential of using large language models (LLMs) as an interactive optimizer for solving maximization problems in a text space using natural language and numerical feedback. Inspired by the classical optimization literature, we classify the natural language feedback into directional and non-directional, where the former is a generalization of the first-order feedback to the natural language space. We find that LLMs are especially capable of optimization when they are provided with {directional feedback}. Based on this insight, we design a new LLM-based optimizer that synthesizes directional feedback from the historical optimization trace to achieve reliable improvement over iterations. Empirically, we show our LLM-based optimizer is more stable and efficient in solving optimization problems, from maximizing mathematical functions to optimizing prompts for writing poems, compared with existing techniques. |
| title | The Importance of Directional Feedback for LLM-based Optimizers |
| topic | Artificial Intelligence Computation and Language Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2405.16434 |