Parallel Context-of-Experts Decoding for Retrieval Augmented Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911372334333952 |
|---|---|
| author | Corallo, Giulio Papotti, Paolo |
| author_facet | Corallo, Giulio Papotti, Paolo |
| contents | Retrieval Augmented Generation faces a trade-off: concatenating documents in a long prompt enables multi-document reasoning but creates prefill bottlenecks, while encoding document KV caches separately offers speed but breaks cross-document interaction. We propose Parallel Context-of-Experts Decoding (Pced), a training-free framework that shifts evidence aggregation from the attention mechanism to the decoding. Pced treats retrieved documents as isolated "experts", synchronizing their predictions via a novel retrieval-aware contrastive decoding rule that weighs expert logits against the model prior. This approach recovers cross-document reasoning capabilities without constructing a shared attention across documents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_08670 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Parallel Context-of-Experts Decoding for Retrieval Augmented Generation Corallo, Giulio Papotti, Paolo Artificial Intelligence Computation and Language Retrieval Augmented Generation faces a trade-off: concatenating documents in a long prompt enables multi-document reasoning but creates prefill bottlenecks, while encoding document KV caches separately offers speed but breaks cross-document interaction. We propose Parallel Context-of-Experts Decoding (Pced), a training-free framework that shifts evidence aggregation from the attention mechanism to the decoding. Pced treats retrieved documents as isolated "experts", synchronizing their predictions via a novel retrieval-aware contrastive decoding rule that weighs expert logits against the model prior. This approach recovers cross-document reasoning capabilities without constructing a shared attention across documents. |
| title | Parallel Context-of-Experts Decoding for Retrieval Augmented Generation |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2601.08670 |