Aligning Query Representation with Rewritten Query and Relevance Judgments in Conversational Search
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916338906169344 |
|---|---|
| author | Mo, Fengran Qu, Chen Mao, Kelong Wu, Yihong Su, Zhan Huang, Kaiyu Nie, Jian-Yun |
| author_facet | Mo, Fengran Qu, Chen Mao, Kelong Wu, Yihong Su, Zhan Huang, Kaiyu Nie, Jian-Yun |
| contents | Conversational search supports multi-turn user-system interactions to solve complex information needs. Different from the traditional single-turn ad-hoc search, conversational search encounters a more challenging problem of context-dependent query understanding with the lengthy and long-tail conversational history context. While conversational query rewriting methods leverage explicit rewritten queries to train a rewriting model to transform the context-dependent query into a stand-stone search query, this is usually done without considering the quality of search results. Conversational dense retrieval methods use fine-tuning to improve a pre-trained ad-hoc query encoder, but they are limited by the conversational search data available for training. In this paper, we leverage both rewritten queries and relevance judgments in the conversational search data to train a better query representation model. The key idea is to align the query representation with those of rewritten queries and relevant documents. The proposed model -- Query Representation Alignment Conversational Dense Retriever, QRACDR, is tested on eight datasets, including various settings in conversational search and ad-hoc search. The results demonstrate the strong performance of QRACDR compared with state-of-the-art methods, and confirm the effectiveness of representation alignment. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_20189 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Aligning Query Representation with Rewritten Query and Relevance Judgments in Conversational Search Mo, Fengran Qu, Chen Mao, Kelong Wu, Yihong Su, Zhan Huang, Kaiyu Nie, Jian-Yun Information Retrieval Computation and Language Conversational search supports multi-turn user-system interactions to solve complex information needs. Different from the traditional single-turn ad-hoc search, conversational search encounters a more challenging problem of context-dependent query understanding with the lengthy and long-tail conversational history context. While conversational query rewriting methods leverage explicit rewritten queries to train a rewriting model to transform the context-dependent query into a stand-stone search query, this is usually done without considering the quality of search results. Conversational dense retrieval methods use fine-tuning to improve a pre-trained ad-hoc query encoder, but they are limited by the conversational search data available for training. In this paper, we leverage both rewritten queries and relevance judgments in the conversational search data to train a better query representation model. The key idea is to align the query representation with those of rewritten queries and relevant documents. The proposed model -- Query Representation Alignment Conversational Dense Retriever, QRACDR, is tested on eight datasets, including various settings in conversational search and ad-hoc search. The results demonstrate the strong performance of QRACDR compared with state-of-the-art methods, and confirm the effectiveness of representation alignment. |
| title | Aligning Query Representation with Rewritten Query and Relevance Judgments in Conversational Search |
| topic | Information Retrieval Computation and Language |
| url | https://arxiv.org/abs/2407.20189 |