Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2606.01482 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918534452346880 |
|---|---|
| author | Vargas, Francielle Robiatti, João Alves, Diego Valem, Lucas Pascotti Seeth, Maximilian Ferrada, Sebastián Agrawal, Ameeta Pedronette, Daniel Freitas, André |
| author_facet | Vargas, Francielle Robiatti, João Alves, Diego Valem, Lucas Pascotti Seeth, Maximilian Ferrada, Sebastián Agrawal, Ameeta Pedronette, Daniel Freitas, André |
| contents | Ensuring factuality and interpretability in RAG remains an open and urgent problem. We introduce Contrastive Evidence Rationale Attention (CERA), the first retrieval framework to employ subjectivity-based hard negative selection and inject an evidential inductive bias into contrastive learning through an auxiliary attention alignment loss. CERA fine-tunes a dense retriever using two training objectives: triplet-based contrastive learning and interpretable attention alignment, which supervises CLS-to-token attention using a part-of-speech-weighted masking distribution over human-annotated factual rationales as evidence signals. Experiments on a large corpus of clinical trial reports demonstrate that the subjectivity-based hard negative selection substantially improves retrieval effectiveness compared to both Contriever and hard negative selection baselines. Furthermore, rationale alignment improves faithfulness while maintaining competitive retrieval performance, supporting the hypothesis that attention can serve as a more faithful explanation of model behavior when guided by human rationales. Moving beyond topical similarity, CERA enables the retriever to identify the specific tokens that constitute supporting evidence, promoting more interpretable evidence selection in RAG systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2606_01482 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Beyond Topical Similarity: Contrastive Evidence Retrieval with Interpretable Attention Alignment in RAG Vargas, Francielle Robiatti, João Alves, Diego Valem, Lucas Pascotti Seeth, Maximilian Ferrada, Sebastián Agrawal, Ameeta Pedronette, Daniel Freitas, André Computation and Language Ensuring factuality and interpretability in RAG remains an open and urgent problem. We introduce Contrastive Evidence Rationale Attention (CERA), the first retrieval framework to employ subjectivity-based hard negative selection and inject an evidential inductive bias into contrastive learning through an auxiliary attention alignment loss. CERA fine-tunes a dense retriever using two training objectives: triplet-based contrastive learning and interpretable attention alignment, which supervises CLS-to-token attention using a part-of-speech-weighted masking distribution over human-annotated factual rationales as evidence signals. Experiments on a large corpus of clinical trial reports demonstrate that the subjectivity-based hard negative selection substantially improves retrieval effectiveness compared to both Contriever and hard negative selection baselines. Furthermore, rationale alignment improves faithfulness while maintaining competitive retrieval performance, supporting the hypothesis that attention can serve as a more faithful explanation of model behavior when guided by human rationales. Moving beyond topical similarity, CERA enables the retriever to identify the specific tokens that constitute supporting evidence, promoting more interpretable evidence selection in RAG systems. |
| title | Beyond Topical Similarity: Contrastive Evidence Retrieval with Interpretable Attention Alignment in RAG |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2606.01482 |