Parallel Key-Value Cache Fusion for Position Invariant RAG
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917900078546944 |
|---|---|
| author | Oh, Philhoon Shin, Jinwoo Thorne, James |
| author_facet | Oh, Philhoon Shin, Jinwoo Thorne, James |
| contents | Recent advancements in Large Language Models (LLMs) underscore the necessity of Retrieval Augmented Generation (RAG) to leverage external information. However, LLMs are sensitive to the position of relevant information within contexts and tend to generate incorrect responses when such information is placed in the middle, known as `Lost in the Middle' phenomenon. In this paper, we introduce a framework that generates consistent outputs for decoder-only models, irrespective of the input context order. Experimental results for three open domain question answering tasks demonstrate position invariance, where the model is not sensitive to input context order, and superior robustness to irrelevent passages compared to prevailing approaches for RAG pipelines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_07523 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Parallel Key-Value Cache Fusion for Position Invariant RAG Oh, Philhoon Shin, Jinwoo Thorne, James Artificial Intelligence Computation and Language Recent advancements in Large Language Models (LLMs) underscore the necessity of Retrieval Augmented Generation (RAG) to leverage external information. However, LLMs are sensitive to the position of relevant information within contexts and tend to generate incorrect responses when such information is placed in the middle, known as `Lost in the Middle' phenomenon. In this paper, we introduce a framework that generates consistent outputs for decoder-only models, irrespective of the input context order. Experimental results for three open domain question answering tasks demonstrate position invariance, where the model is not sensitive to input context order, and superior robustness to irrelevent passages compared to prevailing approaches for RAG pipelines. |
| title | Parallel Key-Value Cache Fusion for Position Invariant RAG |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2501.07523 |