Towards Efficient and Robust Moment Retrieval System: A Unified Framework for Multi-Granularity Models and Temporal Reranking
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916685673398272 |
|---|---|
| author | Tran, Huu-Loc Nguyen-Nhu, Tinh-Anh Phan-Nguyen, Huu-Phong Nguyen, Tien-Huy Nguyen-Dich, Nhat-Minh Dao, Anh Do, Huy-Duc Nguyen, Quan Le, Hoang M. Dinh, Quang-Vinh |
| author_facet | Tran, Huu-Loc Nguyen-Nhu, Tinh-Anh Phan-Nguyen, Huu-Phong Nguyen, Tien-Huy Nguyen-Dich, Nhat-Minh Dao, Anh Do, Huy-Duc Nguyen, Quan Le, Hoang M. Dinh, Quang-Vinh |
| contents | Long-form video understanding presents significant challenges for interactive retrieval systems, as conventional methods struggle to process extensive video content efficiently. Existing approaches often rely on single models, inefficient storage, unstable temporal search, and context-agnostic reranking, limiting their effectiveness. This paper presents a novel framework to enhance interactive video retrieval through four key innovations: (1) an ensemble search strategy that integrates coarse-grained (CLIP) and fine-grained (BEIT3) models to improve retrieval accuracy, (2) a storage optimization technique that reduces redundancy by selecting representative keyframes via TransNetV2 and deduplication, (3) a temporal search mechanism that localizes video segments using dual queries for start and end points, and (4) a temporal reranking approach that leverages neighboring frame context to stabilize rankings. Evaluated on known-item search and question-answering tasks, our framework demonstrates substantial improvements in retrieval precision, efficiency, and user interpretability, offering a robust solution for real-world interactive video retrieval applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_08384 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Efficient and Robust Moment Retrieval System: A Unified Framework for Multi-Granularity Models and Temporal Reranking Tran, Huu-Loc Nguyen-Nhu, Tinh-Anh Phan-Nguyen, Huu-Phong Nguyen, Tien-Huy Nguyen-Dich, Nhat-Minh Dao, Anh Do, Huy-Duc Nguyen, Quan Le, Hoang M. Dinh, Quang-Vinh Computer Vision and Pattern Recognition Long-form video understanding presents significant challenges for interactive retrieval systems, as conventional methods struggle to process extensive video content efficiently. Existing approaches often rely on single models, inefficient storage, unstable temporal search, and context-agnostic reranking, limiting their effectiveness. This paper presents a novel framework to enhance interactive video retrieval through four key innovations: (1) an ensemble search strategy that integrates coarse-grained (CLIP) and fine-grained (BEIT3) models to improve retrieval accuracy, (2) a storage optimization technique that reduces redundancy by selecting representative keyframes via TransNetV2 and deduplication, (3) a temporal search mechanism that localizes video segments using dual queries for start and end points, and (4) a temporal reranking approach that leverages neighboring frame context to stabilize rankings. Evaluated on known-item search and question-answering tasks, our framework demonstrates substantial improvements in retrieval precision, efficiency, and user interpretability, offering a robust solution for real-world interactive video retrieval applications. |
| title | Towards Efficient and Robust Moment Retrieval System: A Unified Framework for Multi-Granularity Models and Temporal Reranking |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.08384 |