EgoCVR: An Egocentric Benchmark for Fine-Grained Composed Video Retrieval
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911965084909568 |
|---|---|
| author | Hummel, Thomas Karthik, Shyamgopal Georgescu, Mariana-Iuliana Akata, Zeynep |
| author_facet | Hummel, Thomas Karthik, Shyamgopal Georgescu, Mariana-Iuliana Akata, Zeynep |
| contents | In Composed Video Retrieval, a video and a textual description which modifies the video content are provided as inputs to the model. The aim is to retrieve the relevant video with the modified content from a database of videos. In this challenging task, the first step is to acquire large-scale training datasets and collect high-quality benchmarks for evaluation. In this work, we introduce EgoCVR, a new evaluation benchmark for fine-grained Composed Video Retrieval using large-scale egocentric video datasets. EgoCVR consists of 2,295 queries that specifically focus on high-quality temporal video understanding. We find that existing Composed Video Retrieval frameworks do not achieve the necessary high-quality temporal video understanding for this task. To address this shortcoming, we adapt a simple training-free method, propose a generic re-ranking framework for Composed Video Retrieval, and demonstrate that this achieves strong results on EgoCVR. Our code and benchmark are freely available at https://github.com/ExplainableML/EgoCVR. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_16658 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | EgoCVR: An Egocentric Benchmark for Fine-Grained Composed Video Retrieval Hummel, Thomas Karthik, Shyamgopal Georgescu, Mariana-Iuliana Akata, Zeynep Computer Vision and Pattern Recognition In Composed Video Retrieval, a video and a textual description which modifies the video content are provided as inputs to the model. The aim is to retrieve the relevant video with the modified content from a database of videos. In this challenging task, the first step is to acquire large-scale training datasets and collect high-quality benchmarks for evaluation. In this work, we introduce EgoCVR, a new evaluation benchmark for fine-grained Composed Video Retrieval using large-scale egocentric video datasets. EgoCVR consists of 2,295 queries that specifically focus on high-quality temporal video understanding. We find that existing Composed Video Retrieval frameworks do not achieve the necessary high-quality temporal video understanding for this task. To address this shortcoming, we adapt a simple training-free method, propose a generic re-ranking framework for Composed Video Retrieval, and demonstrate that this achieves strong results on EgoCVR. Our code and benchmark are freely available at https://github.com/ExplainableML/EgoCVR. |
| title | EgoCVR: An Egocentric Benchmark for Fine-Grained Composed Video Retrieval |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.16658 |