Don't Forget to Connect! Improving RAG with Graph-based Reranking
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910461125984256 |
|---|---|
| author | Dong, Jialin Fatemi, Bahare Perozzi, Bryan Yang, Lin F. Tsitsulin, Anton |
| author_facet | Dong, Jialin Fatemi, Bahare Perozzi, Bryan Yang, Lin F. Tsitsulin, Anton |
| contents | Retrieval Augmented Generation (RAG) has greatly improved the performance of Large Language Model (LLM) responses by grounding generation with context from existing documents. These systems work well when documents are clearly relevant to a question context. But what about when a document has partial information, or less obvious connections to the context? And how should we reason about connections between documents? In this work, we seek to answer these two core questions about RAG generation. We introduce G-RAG, a reranker based on graph neural networks (GNNs) between the retriever and reader in RAG. Our method combines both connections between documents and semantic information (via Abstract Meaning Representation graphs) to provide a context-informed ranker for RAG. G-RAG outperforms state-of-the-art approaches while having smaller computational footprint. Additionally, we assess the performance of PaLM 2 as a reranker and find it to significantly underperform G-RAG. This result emphasizes the importance of reranking for RAG even when using Large Language Models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_18414 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Don't Forget to Connect! Improving RAG with Graph-based Reranking Dong, Jialin Fatemi, Bahare Perozzi, Bryan Yang, Lin F. Tsitsulin, Anton Computation and Language Artificial Intelligence Machine Learning Social and Information Networks Retrieval Augmented Generation (RAG) has greatly improved the performance of Large Language Model (LLM) responses by grounding generation with context from existing documents. These systems work well when documents are clearly relevant to a question context. But what about when a document has partial information, or less obvious connections to the context? And how should we reason about connections between documents? In this work, we seek to answer these two core questions about RAG generation. We introduce G-RAG, a reranker based on graph neural networks (GNNs) between the retriever and reader in RAG. Our method combines both connections between documents and semantic information (via Abstract Meaning Representation graphs) to provide a context-informed ranker for RAG. G-RAG outperforms state-of-the-art approaches while having smaller computational footprint. Additionally, we assess the performance of PaLM 2 as a reranker and find it to significantly underperform G-RAG. This result emphasizes the importance of reranking for RAG even when using Large Language Models. |
| title | Don't Forget to Connect! Improving RAG with Graph-based Reranking |
| topic | Computation and Language Artificial Intelligence Machine Learning Social and Information Networks |
| url | https://arxiv.org/abs/2405.18414 |