Textual Gradients are a Flawed Metaphor for Automatic Prompt Optimization
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909963227496448 |
|---|---|
| author | Melcer, Daniel Chen, Qi Chiang, Wen-Hao Garg, Shweta Garg, Pranav Bock, Christian |
| author_facet | Melcer, Daniel Chen, Qi Chiang, Wen-Hao Garg, Shweta Garg, Pranav Bock, Christian |
| contents | A well-engineered prompt can increase the performance of large language models; automatic prompt optimization techniques aim to increase performance without requiring human effort to tune the prompts. One leading class of prompt optimization techniques introduces the analogy of textual gradients. We investigate the behavior of these textual gradient methods through a series of experiments and case studies. While such methods often result in a performance improvement, our experiments suggest that the gradient analogy does not accurately explain their behavior. Our insights may inform the selection of prompt optimization strategies, and development of new approaches. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_13598 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Textual Gradients are a Flawed Metaphor for Automatic Prompt Optimization Melcer, Daniel Chen, Qi Chiang, Wen-Hao Garg, Shweta Garg, Pranav Bock, Christian Computation and Language Machine Learning A well-engineered prompt can increase the performance of large language models; automatic prompt optimization techniques aim to increase performance without requiring human effort to tune the prompts. One leading class of prompt optimization techniques introduces the analogy of textual gradients. We investigate the behavior of these textual gradient methods through a series of experiments and case studies. While such methods often result in a performance improvement, our experiments suggest that the gradient analogy does not accurately explain their behavior. Our insights may inform the selection of prompt optimization strategies, and development of new approaches. |
| title | Textual Gradients are a Flawed Metaphor for Automatic Prompt Optimization |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2512.13598 |