Augmented Relevance Datasets with Fine-Tuned Small LLMs
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908317483270144 |
|---|---|
| author | Fitte-Rey, Quentin Amrouche, Matyas Deveaud, Romain |
| author_facet | Fitte-Rey, Quentin Amrouche, Matyas Deveaud, Romain |
| contents | Building high-quality datasets and labeling query-document relevance are essential yet resource-intensive tasks, requiring detailed guidelines and substantial effort from human annotators. This paper explores the use of small, fine-tuned large language models (LLMs) to automate relevance assessment, with a focus on improving ranking models' performance by augmenting their training dataset. We fine-tuned small LLMs to enhance relevance assessments, thereby improving dataset creation quality for downstream ranking model training. Our experiments demonstrate that these fine-tuned small LLMs not only outperform certain closed source models on our dataset but also lead to substantial improvements in ranking model performance. These results highlight the potential of leveraging small LLMs for efficient and scalable dataset augmentation, providing a practical solution for search engine optimization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_09816 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Augmented Relevance Datasets with Fine-Tuned Small LLMs Fitte-Rey, Quentin Amrouche, Matyas Deveaud, Romain Information Retrieval Computation and Language H.3.3; I.2.7 Building high-quality datasets and labeling query-document relevance are essential yet resource-intensive tasks, requiring detailed guidelines and substantial effort from human annotators. This paper explores the use of small, fine-tuned large language models (LLMs) to automate relevance assessment, with a focus on improving ranking models' performance by augmenting their training dataset. We fine-tuned small LLMs to enhance relevance assessments, thereby improving dataset creation quality for downstream ranking model training. Our experiments demonstrate that these fine-tuned small LLMs not only outperform certain closed source models on our dataset but also lead to substantial improvements in ranking model performance. These results highlight the potential of leveraging small LLMs for efficient and scalable dataset augmentation, providing a practical solution for search engine optimization. |
| title | Augmented Relevance Datasets with Fine-Tuned Small LLMs |
| topic | Information Retrieval Computation and Language H.3.3; I.2.7 |
| url | https://arxiv.org/abs/2504.09816 |