AugTriever: Unsupervised Dense Retrieval and Domain Adaptation by Scalable Data Augmentation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917822510137344 |
|---|---|
| author | Meng, Rui Liu, Ye Yavuz, Semih Agarwal, Divyansh Tu, Lifu Yu, Ning Zhang, Jianguo Bhat, Meghana Zhou, Yingbo |
| author_facet | Meng, Rui Liu, Ye Yavuz, Semih Agarwal, Divyansh Tu, Lifu Yu, Ning Zhang, Jianguo Bhat, Meghana Zhou, Yingbo |
| contents | Dense retrievers have made significant strides in text retrieval and open-domain question answering. However, most of these achievements have relied heavily on extensive human-annotated supervision. In this study, we aim to develop unsupervised methods for improving dense retrieval models. We propose two approaches that enable annotation-free and scalable training by creating pseudo querydocument pairs: query extraction and transferred query generation. The query extraction method involves selecting salient spans from the original document to generate pseudo queries. On the other hand, the transferred query generation method utilizes generation models trained for other NLP tasks, such as summarization, to produce pseudo queries. Through extensive experimentation, we demonstrate that models trained using these augmentation methods can achieve comparable, if not better, performance than multiple strong dense baselines. Moreover, combining these strategies leads to further improvements, resulting in superior performance of unsupervised dense retrieval, unsupervised domain adaptation and supervised finetuning, benchmarked on both BEIR and ODQA datasets. Code and datasets are publicly available at https://github.com/salesforce/AugTriever. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2212_08841 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | AugTriever: Unsupervised Dense Retrieval and Domain Adaptation by Scalable Data Augmentation Meng, Rui Liu, Ye Yavuz, Semih Agarwal, Divyansh Tu, Lifu Yu, Ning Zhang, Jianguo Bhat, Meghana Zhou, Yingbo Computation and Language Information Retrieval Dense retrievers have made significant strides in text retrieval and open-domain question answering. However, most of these achievements have relied heavily on extensive human-annotated supervision. In this study, we aim to develop unsupervised methods for improving dense retrieval models. We propose two approaches that enable annotation-free and scalable training by creating pseudo querydocument pairs: query extraction and transferred query generation. The query extraction method involves selecting salient spans from the original document to generate pseudo queries. On the other hand, the transferred query generation method utilizes generation models trained for other NLP tasks, such as summarization, to produce pseudo queries. Through extensive experimentation, we demonstrate that models trained using these augmentation methods can achieve comparable, if not better, performance than multiple strong dense baselines. Moreover, combining these strategies leads to further improvements, resulting in superior performance of unsupervised dense retrieval, unsupervised domain adaptation and supervised finetuning, benchmarked on both BEIR and ODQA datasets. Code and datasets are publicly available at https://github.com/salesforce/AugTriever. |
| title | AugTriever: Unsupervised Dense Retrieval and Domain Adaptation by Scalable Data Augmentation |
| topic | Computation and Language Information Retrieval |
| url | https://arxiv.org/abs/2212.08841 |