MultiConIR: Towards multi-condition Information Retrieval
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916932738875392 |
|---|---|
| author | Lu, Xuan Liu, Sifan Yin, Bochao Li, Yongqi Chen, Xinghao Su, Hui Jin, Yaohui Zeng, Wenjun Shen, Xiaoyu |
| author_facet | Lu, Xuan Liu, Sifan Yin, Bochao Li, Yongqi Chen, Xinghao Su, Hui Jin, Yaohui Zeng, Wenjun Shen, Xiaoyu |
| contents | Multi-condition information retrieval (IR) presents a significant, yet underexplored challenge for existing systems. This paper introduces MultiConIR, a benchmark specifically designed to evaluate retrieval and reranking models under nuanced multi-condition query scenarios across five diverse domains. We systematically assess model capabilities through three critical tasks: complexity robustness, relevance monotonicity, and query format sensitivity. Our extensive experiments on 15 models reveal a critical vulnerability: most retrievers and rerankers exhibit severe performance degradation as query complexity increases. Key deficiencies include widespread failure to maintain relevance monotonicity, and high sensitivity to query style and condition placement. The superior performance of GPT-4o reveals the performance gap between IR systems and advanced LLM for handling sophisticated natural language queries. Furthermore, this work delves into the factors contributing to reranker performance deterioration and examines how condition positioning within queries affects similarity assessment, providing crucial insights for advancing IR systems towards complex search scenarios. The code and datasets are available at https://github.com/EIT-NLP/MultiConIR |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_08046 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MultiConIR: Towards multi-condition Information Retrieval Lu, Xuan Liu, Sifan Yin, Bochao Li, Yongqi Chen, Xinghao Su, Hui Jin, Yaohui Zeng, Wenjun Shen, Xiaoyu Information Retrieval Multi-condition information retrieval (IR) presents a significant, yet underexplored challenge for existing systems. This paper introduces MultiConIR, a benchmark specifically designed to evaluate retrieval and reranking models under nuanced multi-condition query scenarios across five diverse domains. We systematically assess model capabilities through three critical tasks: complexity robustness, relevance monotonicity, and query format sensitivity. Our extensive experiments on 15 models reveal a critical vulnerability: most retrievers and rerankers exhibit severe performance degradation as query complexity increases. Key deficiencies include widespread failure to maintain relevance monotonicity, and high sensitivity to query style and condition placement. The superior performance of GPT-4o reveals the performance gap between IR systems and advanced LLM for handling sophisticated natural language queries. Furthermore, this work delves into the factors contributing to reranker performance deterioration and examines how condition positioning within queries affects similarity assessment, providing crucial insights for advancing IR systems towards complex search scenarios. The code and datasets are available at https://github.com/EIT-NLP/MultiConIR |
| title | MultiConIR: Towards multi-condition Information Retrieval |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2503.08046 |