Retrieval-Augmented Generation for Natural Language Processing: A Survey
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914579675611136 |
|---|---|
| author | Wu, Shangyu Xiong, Ying Cui, Yufei Wu, Haolun Chen, Can Yuan, Ye Huang, Lianming Liu, Xue Kuo, Tei-Wei Guan, Nan Xue, Chun Jason |
| author_facet | Wu, Shangyu Xiong, Ying Cui, Yufei Wu, Haolun Chen, Can Yuan, Ye Huang, Lianming Liu, Xue Kuo, Tei-Wei Guan, Nan Xue, Chun Jason |
| contents | Large language models (LLMs) have achieved strong empirical performance in various fields, benefiting from their huge amount of parameters that store knowledge. However, LLMs still suffer from several key issues, such as hallucination problems, knowledge update issues, and lacking domain-specific expertise. The appearance of retrieval-augmented generation (RAG), which leverages an external knowledge base to augment LLMs, mitigates these limitations. This paper presents a systematic review of RAG techniques for natural language processing (NLP), with a focus on retrievers and retrieval fusions. We introduce a novel taxonomy of retrieval fusions, such as query-based, logits-based, latent, and parametric fusion, and provide structured comparisons across accessibility, efficiency, and use cases. The paper further examines RAG applications across diverse NLP tasks, discusses evaluation methodologies and benchmark limitations, and analyzes training paradigms with and without knowledge base updates. Finally, we explore industrial deployment considerations and identify emerging challenges and future directions, including security, efficiency, and graph-based retrieval. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_13193 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Retrieval-Augmented Generation for Natural Language Processing: A Survey Wu, Shangyu Xiong, Ying Cui, Yufei Wu, Haolun Chen, Can Yuan, Ye Huang, Lianming Liu, Xue Kuo, Tei-Wei Guan, Nan Xue, Chun Jason Computation and Language Large language models (LLMs) have achieved strong empirical performance in various fields, benefiting from their huge amount of parameters that store knowledge. However, LLMs still suffer from several key issues, such as hallucination problems, knowledge update issues, and lacking domain-specific expertise. The appearance of retrieval-augmented generation (RAG), which leverages an external knowledge base to augment LLMs, mitigates these limitations. This paper presents a systematic review of RAG techniques for natural language processing (NLP), with a focus on retrievers and retrieval fusions. We introduce a novel taxonomy of retrieval fusions, such as query-based, logits-based, latent, and parametric fusion, and provide structured comparisons across accessibility, efficiency, and use cases. The paper further examines RAG applications across diverse NLP tasks, discusses evaluation methodologies and benchmark limitations, and analyzes training paradigms with and without knowledge base updates. Finally, we explore industrial deployment considerations and identify emerging challenges and future directions, including security, efficiency, and graph-based retrieval. |
| title | Retrieval-Augmented Generation for Natural Language Processing: A Survey |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2407.13193 |