LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912084675002368 |
|---|---|
| author | Xu, Wenda Deutsch, Daniel Finkelstein, Mara Juraska, Juraj Zhang, Biao Liu, Zhongtao Wang, William Yang Li, Lei Freitag, Markus |
| author_facet | Xu, Wenda Deutsch, Daniel Finkelstein, Mara Juraska, Juraj Zhang, Biao Liu, Zhongtao Wang, William Yang Li, Lei Freitag, Markus |
| contents | Recent large language models (LLM) are leveraging human feedback to improve their generation quality. However, human feedback is costly to obtain, especially during inference. In this work, we propose LLMRefine, an inference time optimization method to refine LLM's output. The core idea is to use a learned fine-grained feedback model to pinpoint defects and guide LLM to refine them iteratively. Using original LLM as a proposal of edits, LLMRefine searches for defect-less text via simulated annealing, trading off the exploration and exploitation. We conduct experiments on three text generation tasks, including machine translation, long-form question answering (QA), and topical summarization. LLMRefine consistently outperforms all baseline approaches, achieving improvements up to 1.7 MetricX points on translation tasks, 8.1 ROUGE-L on ASQA, 2.2 ROUGE-L on topical summarization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_09336 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback Xu, Wenda Deutsch, Daniel Finkelstein, Mara Juraska, Juraj Zhang, Biao Liu, Zhongtao Wang, William Yang Li, Lei Freitag, Markus Computation and Language Recent large language models (LLM) are leveraging human feedback to improve their generation quality. However, human feedback is costly to obtain, especially during inference. In this work, we propose LLMRefine, an inference time optimization method to refine LLM's output. The core idea is to use a learned fine-grained feedback model to pinpoint defects and guide LLM to refine them iteratively. Using original LLM as a proposal of edits, LLMRefine searches for defect-less text via simulated annealing, trading off the exploration and exploitation. We conduct experiments on three text generation tasks, including machine translation, long-form question answering (QA), and topical summarization. LLMRefine consistently outperforms all baseline approaches, achieving improvements up to 1.7 MetricX points on translation tasks, 8.1 ROUGE-L on ASQA, 2.2 ROUGE-L on topical summarization. |
| title | LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2311.09336 |