ViDoRe Benchmark V2: Raising the Bar for Visual Retrieval
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912593820516352 |
|---|---|
| author | Macé, Quentin Loison, António Faysse, Manuel |
| author_facet | Macé, Quentin Loison, António Faysse, Manuel |
| contents | The ViDoRe Benchmark V1 was approaching saturation with top models exceeding 90% nDCG@5, limiting its ability to discern improvements. ViDoRe Benchmark V2 introduces realistic, challenging retrieval scenarios via blind contextual querying, long and cross-document queries, and a hybrid synthetic and human-in-the-loop query generation process. It comprises four diverse, multilingual datasets and provides clear evaluation instructions. Initial results demonstrate substantial room for advancement and highlight insights on model generalization and multilingual capability. This benchmark is designed as a living resource, inviting community contributions to maintain relevance through future evaluations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_17166 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ViDoRe Benchmark V2: Raising the Bar for Visual Retrieval Macé, Quentin Loison, António Faysse, Manuel Information Retrieval The ViDoRe Benchmark V1 was approaching saturation with top models exceeding 90% nDCG@5, limiting its ability to discern improvements. ViDoRe Benchmark V2 introduces realistic, challenging retrieval scenarios via blind contextual querying, long and cross-document queries, and a hybrid synthetic and human-in-the-loop query generation process. It comprises four diverse, multilingual datasets and provides clear evaluation instructions. Initial results demonstrate substantial room for advancement and highlight insights on model generalization and multilingual capability. This benchmark is designed as a living resource, inviting community contributions to maintain relevance through future evaluations. |
| title | ViDoRe Benchmark V2: Raising the Bar for Visual Retrieval |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2505.17166 |