Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2409.20075 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929521792385024 |
|---|---|
| author | Guan, Kaisi Cao, Qian Sun, Yuchong Wang, Xiting Song, Ruihua |
| author_facet | Guan, Kaisi Cao, Qian Sun, Yuchong Wang, Xiting Song, Ruihua |
| contents | Retrieval Augmented Generation (RAG) system is important in domains such as e-commerce, which has many long-tail entities and frequently updated information. Most existing works adopt separate modules for retrieval and generation, which may be suboptimal since the retrieval task and the generation task cannot benefit from each other to improve performance. We propose a novel Backbone Shared RAG framework (BSharedRAG). It first uses a domain-specific corpus to continually pre-train a base model as a domain-specific backbone model and then trains two plug-and-play Low-Rank Adaptation (LoRA) modules based on the shared backbone to minimize retrieval and generation losses respectively. Experimental results indicate that our proposed BSharedRAG outperforms baseline models by 5% and 13% in Hit@3 upon two datasets in retrieval evaluation and by 23% in terms of BLEU-3 in generation evaluation. Our codes, models, and dataset are available at https://bsharedrag.github.io. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_20075 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | BSharedRAG: Backbone Shared Retrieval-Augmented Generation for the E-commerce Domain Guan, Kaisi Cao, Qian Sun, Yuchong Wang, Xiting Song, Ruihua Computation and Language Retrieval Augmented Generation (RAG) system is important in domains such as e-commerce, which has many long-tail entities and frequently updated information. Most existing works adopt separate modules for retrieval and generation, which may be suboptimal since the retrieval task and the generation task cannot benefit from each other to improve performance. We propose a novel Backbone Shared RAG framework (BSharedRAG). It first uses a domain-specific corpus to continually pre-train a base model as a domain-specific backbone model and then trains two plug-and-play Low-Rank Adaptation (LoRA) modules based on the shared backbone to minimize retrieval and generation losses respectively. Experimental results indicate that our proposed BSharedRAG outperforms baseline models by 5% and 13% in Hit@3 upon two datasets in retrieval evaluation and by 23% in terms of BLEU-3 in generation evaluation. Our codes, models, and dataset are available at https://bsharedrag.github.io. |
| title | BSharedRAG: Backbone Shared Retrieval-Augmented Generation for the E-commerce Domain |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2409.20075 |