SyNeg: LLM-Driven Synthetic Hard-Negatives for Dense Retrieval
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929644632014848 |
|---|---|
| author | Li, Xiaopeng Li, Xiangyang Zhang, Hao Du, Zhaocheng Jia, Pengyue Wang, Yichao Zhao, Xiangyu Guo, Huifeng Tang, Ruiming |
| author_facet | Li, Xiaopeng Li, Xiangyang Zhang, Hao Du, Zhaocheng Jia, Pengyue Wang, Yichao Zhao, Xiangyu Guo, Huifeng Tang, Ruiming |
| contents | The performance of Dense retrieval (DR) is significantly influenced by the quality of negative sampling. Traditional DR methods primarily depend on naive negative sampling techniques or on mining hard negatives through external retriever and meticulously crafted strategies. However, naive negative sampling often fails to adequately capture the accurate boundaries between positive and negative samples, whereas existing hard negative sampling methods are prone to false negatives, resulting in performance degradation and training instability. Recent advancements in large language models (LLMs) offer an innovative solution to these challenges by generating contextually rich and diverse negative samples. In this work, we present a framework that harnesses LLMs to synthesize high-quality hard negative samples. We first devise a \textit{multi-attribute self-reflection prompting strategy} to direct LLMs in hard negative sample generation. Then, we implement a \textit{hybrid sampling strategy} that integrates these synthetic negatives with traditionally retrieved negatives, thereby stabilizing the training process and improving retrieval performance. Extensive experiments on five benchmark datasets demonstrate the efficacy of our approach, and code is also publicly available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_17250 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SyNeg: LLM-Driven Synthetic Hard-Negatives for Dense Retrieval Li, Xiaopeng Li, Xiangyang Zhang, Hao Du, Zhaocheng Jia, Pengyue Wang, Yichao Zhao, Xiangyu Guo, Huifeng Tang, Ruiming Information Retrieval The performance of Dense retrieval (DR) is significantly influenced by the quality of negative sampling. Traditional DR methods primarily depend on naive negative sampling techniques or on mining hard negatives through external retriever and meticulously crafted strategies. However, naive negative sampling often fails to adequately capture the accurate boundaries between positive and negative samples, whereas existing hard negative sampling methods are prone to false negatives, resulting in performance degradation and training instability. Recent advancements in large language models (LLMs) offer an innovative solution to these challenges by generating contextually rich and diverse negative samples. In this work, we present a framework that harnesses LLMs to synthesize high-quality hard negative samples. We first devise a \textit{multi-attribute self-reflection prompting strategy} to direct LLMs in hard negative sample generation. Then, we implement a \textit{hybrid sampling strategy} that integrates these synthetic negatives with traditionally retrieved negatives, thereby stabilizing the training process and improving retrieval performance. Extensive experiments on five benchmark datasets demonstrate the efficacy of our approach, and code is also publicly available. |
| title | SyNeg: LLM-Driven Synthetic Hard-Negatives for Dense Retrieval |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2412.17250 |