SoftQE: Learned Representations of Queries Expanded by LLMs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911780167483392 |
|---|---|
| author | Pimpalkhute, Varad Heyer, John Yin, Xusen Gupta, Sameer |
| author_facet | Pimpalkhute, Varad Heyer, John Yin, Xusen Gupta, Sameer |
| contents | We investigate the integration of Large Language Models (LLMs) into query encoders to improve dense retrieval without increasing latency and cost, by circumventing the dependency on LLMs at inference time. SoftQE incorporates knowledge from LLMs by mapping embeddings of input queries to those of the LLM-expanded queries. While improvements over various strong baselines on in-domain MS-MARCO metrics are marginal, SoftQE improves performance by 2.83 absolute percentage points on average on five out-of-domain BEIR tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_12663 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SoftQE: Learned Representations of Queries Expanded by LLMs Pimpalkhute, Varad Heyer, John Yin, Xusen Gupta, Sameer Computation and Language Information Retrieval Machine Learning We investigate the integration of Large Language Models (LLMs) into query encoders to improve dense retrieval without increasing latency and cost, by circumventing the dependency on LLMs at inference time. SoftQE incorporates knowledge from LLMs by mapping embeddings of input queries to those of the LLM-expanded queries. While improvements over various strong baselines on in-domain MS-MARCO metrics are marginal, SoftQE improves performance by 2.83 absolute percentage points on average on five out-of-domain BEIR tasks. |
| title | SoftQE: Learned Representations of Queries Expanded by LLMs |
| topic | Computation and Language Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2402.12663 |