Towards Retrieval Augmented Generation over Large Video Libraries
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916296385363968 |
|---|---|
| author | Tevissen, Yannis Guetari, Khalil Petitpont, Frédéric |
| author_facet | Tevissen, Yannis Guetari, Khalil Petitpont, Frédéric |
| contents | Video content creators need efficient tools to repurpose content, a task that often requires complex manual or automated searches. Crafting a new video from large video libraries remains a challenge. In this paper we introduce the task of Video Library Question Answering (VLQA) through an interoperable architecture that applies Retrieval Augmented Generation (RAG) to video libraries. We propose a system that uses large language models (LLMs) to generate search queries, retrieving relevant video moments indexed by speech and visual metadata. An answer generation module then integrates user queries with this metadata to produce responses with specific video timestamps. This approach shows promise in multimedia content retrieval, and AI-assisted video content creation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_14938 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Towards Retrieval Augmented Generation over Large Video Libraries Tevissen, Yannis Guetari, Khalil Petitpont, Frédéric Computation and Language Artificial Intelligence Video content creators need efficient tools to repurpose content, a task that often requires complex manual or automated searches. Crafting a new video from large video libraries remains a challenge. In this paper we introduce the task of Video Library Question Answering (VLQA) through an interoperable architecture that applies Retrieval Augmented Generation (RAG) to video libraries. We propose a system that uses large language models (LLMs) to generate search queries, retrieving relevant video moments indexed by speech and visual metadata. An answer generation module then integrates user queries with this metadata to produce responses with specific video timestamps. This approach shows promise in multimedia content retrieval, and AI-assisted video content creation. |
| title | Towards Retrieval Augmented Generation over Large Video Libraries |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2406.14938 |