Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2410.00857 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917792726384640 |
|---|---|
| author | Ghosh, Reshmi Seetharaman, Rahul Wadhwa, Hitesh Aggarwal, Somyaa Basu, Samyadeep Srinivasan, Soundararajan Zhao, Wenlong Chaudhari, Shreyas Aghazadeh, Ehsan |
| author_facet | Ghosh, Reshmi Seetharaman, Rahul Wadhwa, Hitesh Aggarwal, Somyaa Basu, Samyadeep Srinivasan, Soundararajan Zhao, Wenlong Chaudhari, Shreyas Aghazadeh, Ehsan |
| contents | Retrieval Augmented Generation (RAG) is a widely used approach for leveraging external context in several natural language applications such as question answering and information retrieval. Yet, the exact nature in which a Language Model (LM) leverages this non-parametric memory or retrieved context isn't clearly understood. This paper mechanistically examines the RAG pipeline to highlight that LMs demonstrate a "shortcut'' effect and have a strong bias towards utilizing the retrieved context to answer questions, while relying minimally on model priors. We propose (a) Causal Mediation Analysis; for proving that parametric memory is minimally utilized when answering a question and (b) Attention Contributions and Knockouts for showing the last token residual stream do not get enriched from the subject token in the question, but gets enriched from tokens of RAG-context. We find this pronounced "shortcut'' behaviour to be true across both LLMs (e.g.,LlaMa) and SLMs (e.g., Phi) |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_00857 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Quantifying reliance on external information over parametric knowledge during Retrieval Augmented Generation (RAG) using mechanistic analysis Ghosh, Reshmi Seetharaman, Rahul Wadhwa, Hitesh Aggarwal, Somyaa Basu, Samyadeep Srinivasan, Soundararajan Zhao, Wenlong Chaudhari, Shreyas Aghazadeh, Ehsan Computation and Language Retrieval Augmented Generation (RAG) is a widely used approach for leveraging external context in several natural language applications such as question answering and information retrieval. Yet, the exact nature in which a Language Model (LM) leverages this non-parametric memory or retrieved context isn't clearly understood. This paper mechanistically examines the RAG pipeline to highlight that LMs demonstrate a "shortcut'' effect and have a strong bias towards utilizing the retrieved context to answer questions, while relying minimally on model priors. We propose (a) Causal Mediation Analysis; for proving that parametric memory is minimally utilized when answering a question and (b) Attention Contributions and Knockouts for showing the last token residual stream do not get enriched from the subject token in the question, but gets enriched from tokens of RAG-context. We find this pronounced "shortcut'' behaviour to be true across both LLMs (e.g.,LlaMa) and SLMs (e.g., Phi) |
| title | Quantifying reliance on external information over parametric knowledge during Retrieval Augmented Generation (RAG) using mechanistic analysis |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2410.00857 |