Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.16925 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914237296672768 |
|---|---|
| author | Park, SunYoung Lee, Jong-Hyeon Kim, Youngjune Sung, Daegyu Yu, Younghyun Cha, Young-rok Ju, Jeongho |
| author_facet | Park, SunYoung Lee, Jong-Hyeon Kim, Youngjune Sung, Daegyu Yu, Younghyun Cha, Young-rok Ju, Jeongho |
| contents | We introduce V-Agent, a novel multi-agent platform designed for advanced video search and interactive user-system conversations. By fine-tuning a vision-language model (VLM) with a small video preference dataset and enhancing it with a retrieval vector from an image-text retrieval model, we overcome the limitations of traditional text-based retrieval systems in multimodal scenarios. The VLM-based retrieval model independently embeds video frames and audio transcriptions from an automatic speech recognition (ASR) module into a shared multimodal representation space, enabling V-Agent to interpret both visual and spoken content for context-aware video search. This system consists of three agents-a routing agent, a search agent, and a chat agent-that work collaboratively to address user intents by refining search outputs and communicating with users. The search agent utilizes the VLM-based retrieval model together with an additional re-ranking module to further enhance video retrieval quality. Our proposed framework demonstrates state-of-the-art zero-shot performance on the MultiVENT 2.0 benchmark, highlighting its potential for both academic research and real-world applications. The retrieval model and demo videos are available at https://huggingface.co/NCSOFT/multimodal-embedding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_16925 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | V-Agent: An Interactive Video Search System Using Vision-Language Models Park, SunYoung Lee, Jong-Hyeon Kim, Youngjune Sung, Daegyu Yu, Younghyun Cha, Young-rok Ju, Jeongho Computer Vision and Pattern Recognition Artificial Intelligence Information Retrieval Multiagent Systems We introduce V-Agent, a novel multi-agent platform designed for advanced video search and interactive user-system conversations. By fine-tuning a vision-language model (VLM) with a small video preference dataset and enhancing it with a retrieval vector from an image-text retrieval model, we overcome the limitations of traditional text-based retrieval systems in multimodal scenarios. The VLM-based retrieval model independently embeds video frames and audio transcriptions from an automatic speech recognition (ASR) module into a shared multimodal representation space, enabling V-Agent to interpret both visual and spoken content for context-aware video search. This system consists of three agents-a routing agent, a search agent, and a chat agent-that work collaboratively to address user intents by refining search outputs and communicating with users. The search agent utilizes the VLM-based retrieval model together with an additional re-ranking module to further enhance video retrieval quality. Our proposed framework demonstrates state-of-the-art zero-shot performance on the MultiVENT 2.0 benchmark, highlighting its potential for both academic research and real-world applications. The retrieval model and demo videos are available at https://huggingface.co/NCSOFT/multimodal-embedding. |
| title | V-Agent: An Interactive Video Search System Using Vision-Language Models |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Information Retrieval Multiagent Systems |
| url | https://arxiv.org/abs/2512.16925 |