RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910510192001024 |
|---|---|
| author | Yu, Yue Ping, Wei Liu, Zihan Wang, Boxin You, Jiaxuan Zhang, Chao Shoeybi, Mohammad Catanzaro, Bryan |
| author_facet | Yu, Yue Ping, Wei Liu, Zihan Wang, Boxin You, Jiaxuan Zhang, Chao Shoeybi, Mohammad Catanzaro, Bryan |
| contents | Large language models (LLMs) typically utilize the top-k contexts from a retriever in retrieval-augmented generation (RAG). In this work, we propose a novel instruction fine-tuning framework RankRAG, which instruction-tunes a single LLM for the dual purpose of context ranking and answer generation in RAG. In particular, the instruction-tuned LLMs work surprisingly well by adding a small fraction of ranking data into the training blend, and outperform existing expert ranking models, including the same LLM exclusively fine-tuned on a large amount of ranking data. For generation, we compare our model with many strong baselines, including GPT-4-0613, GPT-4-turbo-2024-0409, and ChatQA-1.5, an open-sourced model with the state-of-the-art performance on RAG benchmarks. Specifically, our Llama3-RankRAG significantly outperforms Llama3-ChatQA-1.5 and GPT-4 models on nine knowledge-intensive benchmarks. In addition, it also performs comparably to GPT-4 on five RAG benchmarks in the biomedical domain without instruction fine-tuning on biomedical data, demonstrating its superb capability for generalization to new domains. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_02485 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs Yu, Yue Ping, Wei Liu, Zihan Wang, Boxin You, Jiaxuan Zhang, Chao Shoeybi, Mohammad Catanzaro, Bryan Computation and Language Artificial Intelligence Information Retrieval Machine Learning Large language models (LLMs) typically utilize the top-k contexts from a retriever in retrieval-augmented generation (RAG). In this work, we propose a novel instruction fine-tuning framework RankRAG, which instruction-tunes a single LLM for the dual purpose of context ranking and answer generation in RAG. In particular, the instruction-tuned LLMs work surprisingly well by adding a small fraction of ranking data into the training blend, and outperform existing expert ranking models, including the same LLM exclusively fine-tuned on a large amount of ranking data. For generation, we compare our model with many strong baselines, including GPT-4-0613, GPT-4-turbo-2024-0409, and ChatQA-1.5, an open-sourced model with the state-of-the-art performance on RAG benchmarks. Specifically, our Llama3-RankRAG significantly outperforms Llama3-ChatQA-1.5 and GPT-4 models on nine knowledge-intensive benchmarks. In addition, it also performs comparably to GPT-4 on five RAG benchmarks in the biomedical domain without instruction fine-tuning on biomedical data, demonstrating its superb capability for generalization to new domains. |
| title | RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs |
| topic | Computation and Language Artificial Intelligence Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2407.02485 |