Unleash the Potential of CLIP for Video Highlight Detection
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913295719464960 |
|---|---|
| author | Han, Donghoon Seo, Seunghyeon Park, Eunhwan Nam, Seong-Uk Kwak, Nojun |
| author_facet | Han, Donghoon Seo, Seunghyeon Park, Eunhwan Nam, Seong-Uk Kwak, Nojun |
| contents | Multimodal and large language models (LLMs) have revolutionized the utilization of open-world knowledge, unlocking novel potentials across various tasks and applications. Among these domains, the video domain has notably benefited from their capabilities. In this paper, we present Highlight-CLIP (HL-CLIP), a method designed to excel in the video highlight detection task by leveraging the pre-trained knowledge embedded in multimodal models. By simply fine-tuning the multimodal encoder in combination with our innovative saliency pooling technique, we have achieved the state-of-the-art performance in the highlight detection task, the QVHighlight Benchmark, to the best of our knowledge. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_01745 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Unleash the Potential of CLIP for Video Highlight Detection Han, Donghoon Seo, Seunghyeon Park, Eunhwan Nam, Seong-Uk Kwak, Nojun Computer Vision and Pattern Recognition Artificial Intelligence Multimodal and large language models (LLMs) have revolutionized the utilization of open-world knowledge, unlocking novel potentials across various tasks and applications. Among these domains, the video domain has notably benefited from their capabilities. In this paper, we present Highlight-CLIP (HL-CLIP), a method designed to excel in the video highlight detection task by leveraging the pre-trained knowledge embedded in multimodal models. By simply fine-tuning the multimodal encoder in combination with our innovative saliency pooling technique, we have achieved the state-of-the-art performance in the highlight detection task, the QVHighlight Benchmark, to the best of our knowledge. |
| title | Unleash the Potential of CLIP for Video Highlight Detection |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2404.01745 |