Learning Refined Document Representations for Dense Retrieval via Deliberate Thinking
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916988947791872 |
|---|---|
| author | Ji, Yifan Xu, Zhipeng Liu, Zhenghao Yan, Yukun Yu, Shi Li, Yishan Liu, Zhiyuan Gu, Yu Yu, Ge Sun, Maosong |
| author_facet | Ji, Yifan Xu, Zhipeng Liu, Zhenghao Yan, Yukun Yu, Shi Li, Yishan Liu, Zhiyuan Gu, Yu Yu, Ge Sun, Maosong |
| contents | Recent dense retrievers increasingly leverage the robust text understanding capabilities of Large Language Models (LLMs), encoding queries and documents into a shared embedding space for effective retrieval. However, most existing methods represent each document with a single embedding, which is less effective at capturing its multifaceted semantics and thereby limits matching accuracy. In this paper, we propose Deliberate Thinking based Dense Retriever (Debater), a novel approach that enhances document representations by incorporating a step-by-step thinking process. Debater introduces a Chain-of-Deliberation mechanism, which iteratively refines document embeddings through a continuous chain-of-thought. To integrate information from various thinking steps, Debater further employs a Self Distillation mechanism that identifies and fuses the most informative steps into a unified embedding. Experimental results show that Debater significantly outperforms existing methods across several retrieval benchmarks, demonstrating superior accuracy and robustness. All codes and datasets are available at https://github.com/OpenBMB/DEBATER. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_12974 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Learning Refined Document Representations for Dense Retrieval via Deliberate Thinking Ji, Yifan Xu, Zhipeng Liu, Zhenghao Yan, Yukun Yu, Shi Li, Yishan Liu, Zhiyuan Gu, Yu Yu, Ge Sun, Maosong Information Retrieval Recent dense retrievers increasingly leverage the robust text understanding capabilities of Large Language Models (LLMs), encoding queries and documents into a shared embedding space for effective retrieval. However, most existing methods represent each document with a single embedding, which is less effective at capturing its multifaceted semantics and thereby limits matching accuracy. In this paper, we propose Deliberate Thinking based Dense Retriever (Debater), a novel approach that enhances document representations by incorporating a step-by-step thinking process. Debater introduces a Chain-of-Deliberation mechanism, which iteratively refines document embeddings through a continuous chain-of-thought. To integrate information from various thinking steps, Debater further employs a Self Distillation mechanism that identifies and fuses the most informative steps into a unified embedding. Experimental results show that Debater significantly outperforms existing methods across several retrieval benchmarks, demonstrating superior accuracy and robustness. All codes and datasets are available at https://github.com/OpenBMB/DEBATER. |
| title | Learning Refined Document Representations for Dense Retrieval via Deliberate Thinking |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2502.12974 |