The Lighthouse of Language: Enhancing LLM Agents via Critique-Guided Improvement
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914112798195712 |
|---|---|
| author | Yang, Ruihan Ye, Fanghua Li, Jian Yuan, Siyu Zhang, Yikai Tu, Zhaopeng Li, Xiaolong Yang, Deqing |
| author_facet | Yang, Ruihan Ye, Fanghua Li, Jian Yuan, Siyu Zhang, Yikai Tu, Zhaopeng Li, Xiaolong Yang, Deqing |
| contents | Large language models (LLMs) have recently transformed from text-based assistants to autonomous agents capable of planning, reasoning, and iteratively improving their actions. While numerical reward signals and verifiers can effectively rank candidate actions, they often provide limited contextual guidance. In contrast, natural language feedback better aligns with the generative capabilities of LLMs, providing richer and more actionable suggestions. However, parsing and implementing this feedback effectively can be challenging for LLM-based agents. In this work, we introduce Critique-Guided Improvement (CGI), a novel two-player framework, comprising an actor model that explores an environment and a critic model that generates detailed nature language feedback. By training the critic to produce fine-grained assessments and actionable revisions, and the actor to utilize these critiques, our approach promotes more robust exploration of alternative strategies while avoiding local optima. Experiments in three interactive environments show that CGI outperforms existing baselines by a substantial margin. Notably, even a small critic model surpasses GPT-4 in feedback quality. The resulting actor achieves state-of-the-art performance, demonstrating the power of explicit iterative guidance to enhance decision-making in LLM-based agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_16024 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Lighthouse of Language: Enhancing LLM Agents via Critique-Guided Improvement Yang, Ruihan Ye, Fanghua Li, Jian Yuan, Siyu Zhang, Yikai Tu, Zhaopeng Li, Xiaolong Yang, Deqing Computation and Language Artificial Intelligence Large language models (LLMs) have recently transformed from text-based assistants to autonomous agents capable of planning, reasoning, and iteratively improving their actions. While numerical reward signals and verifiers can effectively rank candidate actions, they often provide limited contextual guidance. In contrast, natural language feedback better aligns with the generative capabilities of LLMs, providing richer and more actionable suggestions. However, parsing and implementing this feedback effectively can be challenging for LLM-based agents. In this work, we introduce Critique-Guided Improvement (CGI), a novel two-player framework, comprising an actor model that explores an environment and a critic model that generates detailed nature language feedback. By training the critic to produce fine-grained assessments and actionable revisions, and the actor to utilize these critiques, our approach promotes more robust exploration of alternative strategies while avoiding local optima. Experiments in three interactive environments show that CGI outperforms existing baselines by a substantial margin. Notably, even a small critic model surpasses GPT-4 in feedback quality. The resulting actor achieves state-of-the-art performance, demonstrating the power of explicit iterative guidance to enhance decision-making in LLM-based agents. |
| title | The Lighthouse of Language: Enhancing LLM Agents via Critique-Guided Improvement |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2503.16024 |