Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.09930 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915721536077824 |
|---|---|
| author | Zhu, Zixiao Zhou, Hanzhang Feng, Zijian Li, Tianjiao Deryl, Chua Jia Jim Onn, Mak Lee Ng, Gee Wah Mao, Kezhi |
| author_facet | Zhu, Zixiao Zhou, Hanzhang Feng, Zijian Li, Tianjiao Deryl, Chua Jia Jim Onn, Mak Lee Ng, Gee Wah Mao, Kezhi |
| contents | Prompt optimization (PO) provides a practical way to improve response quality when users lack the time or expertise to manually craft effective prompts. Existing methods typically rely on LLMs' self-generation ability to optimize prompts. However, due to limited downward compatibility, the instruction-heavy prompts generated by advanced LLMs can overwhelm lightweight inference models and degrade response quality, while also lacking interpretability due to implicit optimization. In this work, we rethink prompt optimization through the lens of explicit and interpretable design. We first identify a set of model-agnostic prompt quality merits and empirically validate their effectiveness in enhancing prompt and response quality. We then introduce MePO, a merit-guided, locally deployable prompt optimizer trained on our merit-guided prompt preference dataset generated by a lightweight LLM. MePO avoids online optimization, reduces privacy concerns, and, by learning clear, interpretable merits, generalizes effectively to both large-scale and lightweight inference models. Experiments demonstrate that MePO achieves better results across diverse tasks and model types, offering a scalable and robust solution for real-world deployment. The code, model and dataset can be found in https://github.com/MidiyaZhu/MePO |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_09930 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Rethinking Prompt Optimizers: From Prompt Merits to Optimization Zhu, Zixiao Zhou, Hanzhang Feng, Zijian Li, Tianjiao Deryl, Chua Jia Jim Onn, Mak Lee Ng, Gee Wah Mao, Kezhi Computation and Language Prompt optimization (PO) provides a practical way to improve response quality when users lack the time or expertise to manually craft effective prompts. Existing methods typically rely on LLMs' self-generation ability to optimize prompts. However, due to limited downward compatibility, the instruction-heavy prompts generated by advanced LLMs can overwhelm lightweight inference models and degrade response quality, while also lacking interpretability due to implicit optimization. In this work, we rethink prompt optimization through the lens of explicit and interpretable design. We first identify a set of model-agnostic prompt quality merits and empirically validate their effectiveness in enhancing prompt and response quality. We then introduce MePO, a merit-guided, locally deployable prompt optimizer trained on our merit-guided prompt preference dataset generated by a lightweight LLM. MePO avoids online optimization, reduces privacy concerns, and, by learning clear, interpretable merits, generalizes effectively to both large-scale and lightweight inference models. Experiments demonstrate that MePO achieves better results across diverse tasks and model types, offering a scalable and robust solution for real-world deployment. The code, model and dataset can be found in https://github.com/MidiyaZhu/MePO |
| title | Rethinking Prompt Optimizers: From Prompt Merits to Optimization |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2505.09930 |