Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917777870159872 |
|---|---|
| author | Weller, Orion Van Durme, Benjamin Lawrie, Dawn Paranjape, Ashwin Zhang, Yuhao Hessel, Jack |
| author_facet | Weller, Orion Van Durme, Benjamin Lawrie, Dawn Paranjape, Ashwin Zhang, Yuhao Hessel, Jack |
| contents | Instruction-tuned language models (LM) are able to respond to imperative commands, providing a more natural user interface compared to their base counterparts. In this work, we present Promptriever, the first retrieval model able to be prompted like an LM. To train Promptriever, we curate and release a new instance-level instruction training set from MS MARCO, spanning nearly 500k instances. Promptriever not only achieves strong performance on standard retrieval tasks, but also follows instructions. We observe: (1) large gains (reaching SoTA) on following detailed relevance instructions (+14.3 p-MRR / +3.1 nDCG on FollowIR), (2) significantly increased robustness to lexical choices/phrasing in the query+instruction (+12.9 Robustness@10 on InstructIR), and (3) the ability to perform hyperparameter search via prompting to reliably improve retrieval performance (+1.4 average increase on BEIR). Promptriever demonstrates that retrieval models can be controlled with prompts on a per-query basis, setting the stage for future work aligning LM prompting techniques with information retrieval. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_11136 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Models Weller, Orion Van Durme, Benjamin Lawrie, Dawn Paranjape, Ashwin Zhang, Yuhao Hessel, Jack Information Retrieval Computation and Language Machine Learning Instruction-tuned language models (LM) are able to respond to imperative commands, providing a more natural user interface compared to their base counterparts. In this work, we present Promptriever, the first retrieval model able to be prompted like an LM. To train Promptriever, we curate and release a new instance-level instruction training set from MS MARCO, spanning nearly 500k instances. Promptriever not only achieves strong performance on standard retrieval tasks, but also follows instructions. We observe: (1) large gains (reaching SoTA) on following detailed relevance instructions (+14.3 p-MRR / +3.1 nDCG on FollowIR), (2) significantly increased robustness to lexical choices/phrasing in the query+instruction (+12.9 Robustness@10 on InstructIR), and (3) the ability to perform hyperparameter search via prompting to reliably improve retrieval performance (+1.4 average increase on BEIR). Promptriever demonstrates that retrieval models can be controlled with prompts on a per-query basis, setting the stage for future work aligning LM prompting techniques with information retrieval. |
| title | Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Models |
| topic | Information Retrieval Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2409.11136 |