Simplified Longitudinal Retrieval Experiments: A Case Study on Query Expansion and Document Boosting
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908552042381312 |
|---|---|
| author | Keller, Jüri Fröbe, Maik Hendriksen, Gijs Alexander, Daria Potthast, Martin Schaer, Philipp |
| author_facet | Keller, Jüri Fröbe, Maik Hendriksen, Gijs Alexander, Daria Potthast, Martin Schaer, Philipp |
| contents | The longitudinal evaluation of retrieval systems aims to capture how information needs and documents evolve over time. However, classical Cranfield-style retrieval evaluations only consist of a static set of queries and documents and thereby miss time as an evaluation dimension. Therefore, longitudinal evaluations need to complement retrieval toolkits with custom logic. This custom logic increases the complexity of research software, which might reduce the reproducibility and extensibility of experiments. Based on our submissions to the 2024 edition of LongEval, we propose a custom extension of ir_datasets for longitudinal retrieval experiments. This extension allows for declaratively, instead of imperatively, describing important aspects of longitudinal retrieval experiments, e.g., which queries, documents, and/or relevance feedback are available at which point in time. We reimplement our submissions to LongEval 2024 against our new ir_datasets extension, and find that the declarative access can reduce the complexity of the code. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_17440 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Simplified Longitudinal Retrieval Experiments: A Case Study on Query Expansion and Document Boosting Keller, Jüri Fröbe, Maik Hendriksen, Gijs Alexander, Daria Potthast, Martin Schaer, Philipp Information Retrieval The longitudinal evaluation of retrieval systems aims to capture how information needs and documents evolve over time. However, classical Cranfield-style retrieval evaluations only consist of a static set of queries and documents and thereby miss time as an evaluation dimension. Therefore, longitudinal evaluations need to complement retrieval toolkits with custom logic. This custom logic increases the complexity of research software, which might reduce the reproducibility and extensibility of experiments. Based on our submissions to the 2024 edition of LongEval, we propose a custom extension of ir_datasets for longitudinal retrieval experiments. This extension allows for declaratively, instead of imperatively, describing important aspects of longitudinal retrieval experiments, e.g., which queries, documents, and/or relevance feedback are available at which point in time. We reimplement our submissions to LongEval 2024 against our new ir_datasets extension, and find that the declarative access can reduce the complexity of the code. |
| title | Simplified Longitudinal Retrieval Experiments: A Case Study on Query Expansion and Document Boosting |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2509.17440 |