Cover Image

The Atlas of In-Context Learning: How Attention Heads Shape In-Context Retrieval Augmentation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kahardipraja, Patrick, Achtibat, Reduan, Wiegand, Thomas, Samek, Wojciech, Lapuschkin, Sebastian
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Information Retrieval Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Attribution-Guided Pruning for Insight and Control: Circuit Discovery and Targeted Correction in Small-scale LLMs
by: Hatefi, Sayed Mohammad Vakilzadeh, et al.
Published: (2025)

AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformers
by: Achtibat, Reduan, et al.
Published: (2024)

Attribution-Guided Decoding
by: Komorowski, Piotr, et al.
Published: (2025)

FADE: Why Bad Descriptions Happen to Good Features
by: Puri, Bruno, et al.
Published: (2025)

Pruning By Explaining Revisited: Optimizing Attribution Methods to Prune CNNs and Transformers
by: Hatefi, Sayed Mohammad Vakilzadeh, et al.
Published: (2024)

From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagation
by: Achtibat, Reduan, et al.
Published: (2022)

Optimization of Retrieval-Augmented Generation Context with Outlier Detection
by: Bulgakov, Vitaly
Published: (2024)

GRAM: Generative Retrieval Augmented Matching of Data Schemas in the Context of Data Security
by: Liu, Xuanqing, et al.
Published: (2024)

RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs
by: Yu, Yue, et al.
Published: (2024)

Self-Augmented In-Context Learning for Unsupervised Word Translation
by: Li, Yaoyiran, et al.
Published: (2024)

SPARC-RAG: Adaptive Sequential-Parallel Scaling with Context Management for Retrieval-Augmented Generation
by: Yang, Yuxin, et al.
Published: (2026)

Retrieval-Augmented Generation as Noisy In-Context Learning: A Unified Theory and Risk Bounds
by: Guo, Yang, et al.
Published: (2025)

Understanding the (Extra-)Ordinary: Validating Deep Model Decisions with Prototypical Concept-based Explanations
by: Dreyer, Maximilian, et al.
Published: (2023)

Atlas-Alignment: Making Interpretability Transferable Across Language Models
by: Puri, Bruno, et al.
Published: (2025)

Retrieval meets Long Context Large Language Models
by: Xu, Peng, et al.
Published: (2023)

ACER: Automatic Language Model Context Extension via Retrieval
by: Gao, Luyu, et al.
Published: (2024)

Retrieval-Augmented Generation with Graphs (GraphRAG)
by: Han, Haoyu, et al.
Published: (2024)

Accelerating Retrieval-Augmented Language Model Serving with Speculation
by: Zhang, Zhihao, et al.
Published: (2024)

Cluster-based Adaptive Retrieval: Dynamic Context Selection for RAG Applications
by: Xu, Yifan, et al.
Published: (2025)

Context Retrieval via Normalized Contextual Latent Interaction for Conversational Agent
by: Liu, Junfeng, et al.
Published: (2023)

Context Awareness Gate For Retrieval Augmented Generation
by: Heydari, Mohammad Hassan, et al.
Published: (2024)

GraphRAFT: Retrieval Augmented Fine-Tuning for Knowledge Graphs on Graph Databases
by: Clemedtson, Alfred, et al.
Published: (2025)

Context Is Not Comprehension
by: Pan, Alex, et al.
Published: (2025)

GraphER: An Efficient Graph-Based Enrichment and Reranking Method for Retrieval-Augmented Generation
by: Miao, Ruizhong, et al.
Published: (2026)

Stochastic RAG: End-to-End Retrieval-Augmented Generation through Expected Utility Maximization
by: Zamani, Hamed, et al.
Published: (2024)

Contradictions in Context: Challenges for Retrieval-Augmented Generation in Healthcare
by: Javadi, Saeedeh, et al.
Published: (2025)

CoMAC: Conversational Agent for Multi-Source Auxiliary Context with Sparse and Symmetric Latent Interactions
by: Liu, Junfeng, et al.
Published: (2025)

RARe: Retrieval Augmented Retrieval with In-Context Examples
by: Tejaswi, Atula, et al.
Published: (2024)

NANOGPT: A Query-Driven Large Language Model Retrieval-Augmented Generation System for Nanotechnology Research
by: Chandrasekhar, Achuth, et al.
Published: (2025)

Simple Is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation
by: Li, Mufei, et al.
Published: (2024)

GORAG: Graph-based Online Retrieval Augmented Generation for Dynamic Few-shot Social Media Text Classification
by: Wang, Yubo, et al.
Published: (2025)

Large Language Model Augmented Exercise Retrieval for Personalized Language Learning
by: Xu, Austin, et al.
Published: (2024)

Retrieval-Enhanced Machine Learning: Synthesis and Opportunities
by: Kim, To Eun, et al.
Published: (2024)

Towards Mixed-Modal Retrieval for Universal Retrieval-Augmented Generation
by: Zhang, Chenghao, et al.
Published: (2025)

Scaling Laws for Many-Shot In-Context Learning with Self-Generated Annotations
by: Gu, Zhengyao, et al.
Published: (2025)

A Close Look at Decomposition-based XAI-Methods for Transformer Language Models
by: Arras, Leila, et al.
Published: (2025)

Sparse and Dense Retrievers Learn Better Together: Joint Sparse-Dense Optimization for Text-Image Retrieval
by: Song, Jonghyun, et al.
Published: (2025)

Re-ranking the Context for Multimodal Retrieval Augmented Generation
by: Mortaheb, Matin, et al.
Published: (2025)

XProvence: Zero-Cost Multilingual Context Pruning for Retrieval-Augmented Generation
by: Mohamed, Youssef, et al.
Published: (2026)

CoRAG: Collaborative Retrieval-Augmented Generation
by: Muhamed, Aashiq, et al.
Published: (2025)