Re-Ex: Revising after Explanation Reduces the Factual Errors in LLM Responses
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913788965421056 |
|---|---|
| author | Kim, Juyeon Lee, Jeongeun Chang, Yoonho Choi, Chanyeol Kim, Junseong Sohn, Jy-yong |
| author_facet | Kim, Juyeon Lee, Jeongeun Chang, Yoonho Choi, Chanyeol Kim, Junseong Sohn, Jy-yong |
| contents | Mitigating hallucination issues is a key challenge that must be overcome to reliably deploy large language models (LLMs) in real-world scenarios. Recently, various methods have been proposed to detect and revise factual errors in LLM-generated texts, in order to reduce hallucination. In this paper, we propose Re-Ex, a method for post-editing LLM-generated responses. Re-Ex introduces a novel reasoning step dubbed as the factual error explanation step. Re-Ex revises the initial response of LLMs using 3-steps : first, external tools are used to retrieve the evidences of the factual errors in the initial LLM response; next, LLM is instructed to explain the problematic parts of the response based on the gathered evidence; finally, LLM revises the initial response using the explanations provided in the previous step. In addition to the explanation step, Re-Ex also incorporates new prompting techniques to reduce the token count and inference time required for the response revision process. Compared with existing methods including FacTool, CoVE, and RARR, Re-Ex provides better detection and revision performance with less inference time and fewer tokens in multiple benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_17097 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Re-Ex: Revising after Explanation Reduces the Factual Errors in LLM Responses Kim, Juyeon Lee, Jeongeun Chang, Yoonho Choi, Chanyeol Kim, Junseong Sohn, Jy-yong Computation and Language Artificial Intelligence Mitigating hallucination issues is a key challenge that must be overcome to reliably deploy large language models (LLMs) in real-world scenarios. Recently, various methods have been proposed to detect and revise factual errors in LLM-generated texts, in order to reduce hallucination. In this paper, we propose Re-Ex, a method for post-editing LLM-generated responses. Re-Ex introduces a novel reasoning step dubbed as the factual error explanation step. Re-Ex revises the initial response of LLMs using 3-steps : first, external tools are used to retrieve the evidences of the factual errors in the initial LLM response; next, LLM is instructed to explain the problematic parts of the response based on the gathered evidence; finally, LLM revises the initial response using the explanations provided in the previous step. In addition to the explanation step, Re-Ex also incorporates new prompting techniques to reduce the token count and inference time required for the response revision process. Compared with existing methods including FacTool, CoVE, and RARR, Re-Ex provides better detection and revision performance with less inference time and fewer tokens in multiple benchmarks. |
| title | Re-Ex: Revising after Explanation Reduces the Factual Errors in LLM Responses |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2402.17097 |