MA-SAPO: Multi-Agent Reasoning for Score-Aware Prompt Optimization
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908922651082752 |
|---|---|
| author | Seo, Wonduk Lee, Juhyeon Koh, Junseo Choi, Wonseok An, Hyunjin Park, Jian lee, Seunghyun Chen, Haihua Bu, Yi |
| author_facet | Seo, Wonduk Lee, Juhyeon Koh, Junseo Choi, Wonseok An, Hyunjin Park, Jian lee, Seunghyun Chen, Haihua Bu, Yi |
| contents | Prompt optimization has become a practical way to improve the performance of Large Language Models (LLMs) without retraining. However, most existing frameworks treat evaluation as a black box, relying solely on outcome scores without explaining why prompts succeed or fail. Moreover, they involve repetitive trial-and-error refinements that remain implicit, offering limited interpretability or actionable guidance for systematic improvement. In this paper, we propose MA-SAPO: a new Multi-Agent Reasoning for Score Aware Prompt Optimization framework that links evaluation outcomes directly to targeted refinements. Specifically, in the Training Phase, multiple agents interpret evaluation scores, diagnose weaknesses, and generate concrete revision directives, which are stored as reusable reasoning assets. In the Test Phase, an analyzer agent retrieves relevant exemplars and assets for a new prompt, and a refiner agent applies evidence-based edits to improve the prompt and its response. By grounding optimization in structured reasoning, MA-SAPO ensures edits are interpretable, auditable, and controllable. Experiments on the HelpSteer1/2 benchmarks show that our framework consistently outperforms single-pass prompting, retrieval-augmented generation, and prior multi-agent methods across multiple evaluation metrics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_16635 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MA-SAPO: Multi-Agent Reasoning for Score-Aware Prompt Optimization Seo, Wonduk Lee, Juhyeon Koh, Junseo Choi, Wonseok An, Hyunjin Park, Jian lee, Seunghyun Chen, Haihua Bu, Yi Multiagent Systems Artificial Intelligence Computation and Language Human-Computer Interaction Information Retrieval Prompt optimization has become a practical way to improve the performance of Large Language Models (LLMs) without retraining. However, most existing frameworks treat evaluation as a black box, relying solely on outcome scores without explaining why prompts succeed or fail. Moreover, they involve repetitive trial-and-error refinements that remain implicit, offering limited interpretability or actionable guidance for systematic improvement. In this paper, we propose MA-SAPO: a new Multi-Agent Reasoning for Score Aware Prompt Optimization framework that links evaluation outcomes directly to targeted refinements. Specifically, in the Training Phase, multiple agents interpret evaluation scores, diagnose weaknesses, and generate concrete revision directives, which are stored as reusable reasoning assets. In the Test Phase, an analyzer agent retrieves relevant exemplars and assets for a new prompt, and a refiner agent applies evidence-based edits to improve the prompt and its response. By grounding optimization in structured reasoning, MA-SAPO ensures edits are interpretable, auditable, and controllable. Experiments on the HelpSteer1/2 benchmarks show that our framework consistently outperforms single-pass prompting, retrieval-augmented generation, and prior multi-agent methods across multiple evaluation metrics. |
| title | MA-SAPO: Multi-Agent Reasoning for Score-Aware Prompt Optimization |
| topic | Multiagent Systems Artificial Intelligence Computation and Language Human-Computer Interaction Information Retrieval |
| url | https://arxiv.org/abs/2510.16635 |