Which Neurons Matter in IR? Applying Integrated Gradients-based Methods to Understand Cross-Encoders
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929410179858432 |
|---|---|
| author | Vast, Mathias Van Cooten, Basile Soulier, Laure Piwowarski, Benjamin |
| author_facet | Vast, Mathias Van Cooten, Basile Soulier, Laure Piwowarski, Benjamin |
| contents | With the recent addition of Retrieval-Augmented Generation (RAG), the scope and importance of Information Retrieval (IR) has expanded. As a result, the importance of a deeper understanding of IR models also increases. However, interpretability in IR remains under-explored, especially when it comes to the models' inner mechanisms. In this paper, we explore the possibility of adapting Integrated Gradient-based methods in an IR context to identify the role of individual neurons within the model. In particular, we provide new insights into the role of what we call "relevance" neurons, as well as how they deal with unseen data. Finally, we carry out an in-depth pruning study to validate our findings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_19309 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Which Neurons Matter in IR? Applying Integrated Gradients-based Methods to Understand Cross-Encoders Vast, Mathias Van Cooten, Basile Soulier, Laure Piwowarski, Benjamin Information Retrieval With the recent addition of Retrieval-Augmented Generation (RAG), the scope and importance of Information Retrieval (IR) has expanded. As a result, the importance of a deeper understanding of IR models also increases. However, interpretability in IR remains under-explored, especially when it comes to the models' inner mechanisms. In this paper, we explore the possibility of adapting Integrated Gradient-based methods in an IR context to identify the role of individual neurons within the model. In particular, we provide new insights into the role of what we call "relevance" neurons, as well as how they deal with unseen data. Finally, we carry out an in-depth pruning study to validate our findings. |
| title | Which Neurons Matter in IR? Applying Integrated Gradients-based Methods to Understand Cross-Encoders |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2406.19309 |