Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2512.20854 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918486543958016 |
|---|---|
| author | Schwartz, Shelly Vasilyev, Oleg Sawaya, Randy |
| author_facet | Schwartz, Shelly Vasilyev, Oleg Sawaya, Randy |
| contents | In realistic retrieval settings with large and evolving knowledge bases, the total number of documents relevant to a query is typically unknown, and recall cannot be computed. In this paper, we evaluate several established strategies for handling this limitation by measuring the correlation between retrieval quality metrics and LLM-based judgments of response quality, where responses are generated from the retrieved documents. We conduct experiments across multiple datasets with a relatively low number of relevant documents (2-15). We also introduce a simple retrieval quality measure that performs well without requiring knowledge of the total number of relevant documents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_20854 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | How important is Recall for Measuring Retrieval Quality? Schwartz, Shelly Vasilyev, Oleg Sawaya, Randy Computation and Language Information Retrieval In realistic retrieval settings with large and evolving knowledge bases, the total number of documents relevant to a query is typically unknown, and recall cannot be computed. In this paper, we evaluate several established strategies for handling this limitation by measuring the correlation between retrieval quality metrics and LLM-based judgments of response quality, where responses are generated from the retrieved documents. We conduct experiments across multiple datasets with a relatively low number of relevant documents (2-15). We also introduce a simple retrieval quality measure that performs well without requiring knowledge of the total number of relevant documents. |
| title | How important is Recall for Measuring Retrieval Quality? |
| topic | Computation and Language Information Retrieval |
| url | https://arxiv.org/abs/2512.20854 |