Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.24334 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908995356196864 |
|---|---|
| author | Berdyugina, Daria Cohen, Anaëlle Rioual, Yohann |
| author_facet | Berdyugina, Daria Cohen, Anaëlle Rioual, Yohann |
| contents | Standard Retrieval-Augmented Generation (RAG) chunking methods often create excessive redundancy, increasing storage costs and slowing retrieval. This study explores chunk filtering strategies, such as semantic, topic-based, and named-entity-based methods in order to reduce the indexed corpus while preserving retrieval quality. Experiments are conducted on multiple corpora. Retrieval performance is evaluated using a token-based framework based on precision, recall, and intersection-over-union metrics. Results indicate that entity-based filtering can reduce vector index size by approximately 25% to 36% while maintaining high retrieval quality close to the baseline. These findings suggest that redundancy introduced during chunking can be effectively reduced through lightweight filtering, improving the efficiency of retrieval-oriented components in RAG pipelines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_24334 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Reducing Redundancy in Retrieval-Augmented Generation through Chunk Filtering Berdyugina, Daria Cohen, Anaëlle Rioual, Yohann Computation and Language Standard Retrieval-Augmented Generation (RAG) chunking methods often create excessive redundancy, increasing storage costs and slowing retrieval. This study explores chunk filtering strategies, such as semantic, topic-based, and named-entity-based methods in order to reduce the indexed corpus while preserving retrieval quality. Experiments are conducted on multiple corpora. Retrieval performance is evaluated using a token-based framework based on precision, recall, and intersection-over-union metrics. Results indicate that entity-based filtering can reduce vector index size by approximately 25% to 36% while maintaining high retrieval quality close to the baseline. These findings suggest that redundancy introduced during chunking can be effectively reduced through lightweight filtering, improving the efficiency of retrieval-oriented components in RAG pipelines. |
| title | Reducing Redundancy in Retrieval-Augmented Generation through Chunk Filtering |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2604.24334 |