HeteRAG: A Heterogeneous Retrieval-augmented Generation Framework with Decoupled Knowledge Representations
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917985691631616 |
|---|---|
| author | Yang, Peiru Li, Xintian Hu, Zhiyang Wang, Jiapeng Yin, Jinhua Wang, Huili He, Lizhi Yang, Shuai Wang, Shangguang Huang, Yongfeng Qi, Tao |
| author_facet | Yang, Peiru Li, Xintian Hu, Zhiyang Wang, Jiapeng Yin, Jinhua Wang, Huili He, Lizhi Yang, Shuai Wang, Shangguang Huang, Yongfeng Qi, Tao |
| contents | Retrieval-augmented generation (RAG) methods can enhance the performance of LLMs by incorporating retrieved knowledge chunks into the generation process. In general, the retrieval and generation steps usually have different requirements for these knowledge chunks. The retrieval step benefits from comprehensive information to improve retrieval accuracy, whereas excessively long chunks may introduce redundant contextual information, thereby diminishing both the effectiveness and efficiency of the generation process. However, existing RAG methods typically employ identical representations of knowledge chunks for both retrieval and generation, resulting in suboptimal performance. In this paper, we propose a heterogeneous RAG framework (\myname) that decouples the representations of knowledge chunks for retrieval and generation, thereby enhancing the LLMs in both effectiveness and efficiency. Specifically, we utilize short chunks to represent knowledge to adapt the generation step and utilize the corresponding chunk with its contextual information from multi-granular views to enhance retrieval accuracy. We further introduce an adaptive prompt tuning method for the retrieval model to adapt the heterogeneous retrieval augmented generation process. Extensive experiments demonstrate that \myname achieves significant improvements compared to baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_10529 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | HeteRAG: A Heterogeneous Retrieval-augmented Generation Framework with Decoupled Knowledge Representations Yang, Peiru Li, Xintian Hu, Zhiyang Wang, Jiapeng Yin, Jinhua Wang, Huili He, Lizhi Yang, Shuai Wang, Shangguang Huang, Yongfeng Qi, Tao Information Retrieval Artificial Intelligence Retrieval-augmented generation (RAG) methods can enhance the performance of LLMs by incorporating retrieved knowledge chunks into the generation process. In general, the retrieval and generation steps usually have different requirements for these knowledge chunks. The retrieval step benefits from comprehensive information to improve retrieval accuracy, whereas excessively long chunks may introduce redundant contextual information, thereby diminishing both the effectiveness and efficiency of the generation process. However, existing RAG methods typically employ identical representations of knowledge chunks for both retrieval and generation, resulting in suboptimal performance. In this paper, we propose a heterogeneous RAG framework (\myname) that decouples the representations of knowledge chunks for retrieval and generation, thereby enhancing the LLMs in both effectiveness and efficiency. Specifically, we utilize short chunks to represent knowledge to adapt the generation step and utilize the corresponding chunk with its contextual information from multi-granular views to enhance retrieval accuracy. We further introduce an adaptive prompt tuning method for the retrieval model to adapt the heterogeneous retrieval augmented generation process. Extensive experiments demonstrate that \myname achieves significant improvements compared to baselines. |
| title | HeteRAG: A Heterogeneous Retrieval-augmented Generation Framework with Decoupled Knowledge Representations |
| topic | Information Retrieval Artificial Intelligence |
| url | https://arxiv.org/abs/2504.10529 |