Coarse-Tuning for Ad-hoc Document Retrieval Using Pre-trained Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916179299270656 |
|---|---|
| author | Keyaki, Atsushi Keyaki, Ribeka |
| author_facet | Keyaki, Atsushi Keyaki, Ribeka |
| contents | Fine-tuning in information retrieval systems using pre-trained language models (PLM-based IR) requires learning query representations and query-document relations, in addition to downstream task-specific learning. This study introduces coarse-tuning as an intermediate learning stage that bridges pre-training and fine-tuning. By learning query representations and query-document relations in coarse-tuning, we aim to reduce the load of fine-tuning and improve the learning effect of downstream IR tasks. We propose Query-Document Pair Prediction (QDPP) for coarse-tuning, which predicts the appropriateness of query-document pairs. Evaluation experiments show that the proposed method significantly improves MRR and/or nDCG@5 in four ad-hoc document retrieval datasets. Furthermore, the results of the query prediction task suggested that coarse-tuning facilitated learning of query representation and query-document relations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_16915 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Coarse-Tuning for Ad-hoc Document Retrieval Using Pre-trained Language Models Keyaki, Atsushi Keyaki, Ribeka Information Retrieval Artificial Intelligence Computation and Language Machine Learning Fine-tuning in information retrieval systems using pre-trained language models (PLM-based IR) requires learning query representations and query-document relations, in addition to downstream task-specific learning. This study introduces coarse-tuning as an intermediate learning stage that bridges pre-training and fine-tuning. By learning query representations and query-document relations in coarse-tuning, we aim to reduce the load of fine-tuning and improve the learning effect of downstream IR tasks. We propose Query-Document Pair Prediction (QDPP) for coarse-tuning, which predicts the appropriateness of query-document pairs. Evaluation experiments show that the proposed method significantly improves MRR and/or nDCG@5 in four ad-hoc document retrieval datasets. Furthermore, the results of the query prediction task suggested that coarse-tuning facilitated learning of query representation and query-document relations. |
| title | Coarse-Tuning for Ad-hoc Document Retrieval Using Pre-trained Language Models |
| topic | Information Retrieval Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2403.16915 |