Facilitating Video Story Interaction with Multi-Agent Collaborative System
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910929946411008 |
|---|---|
| author | Zhang, Yiwen Hao, Jianing Wang, Zhan Sheng, Hongling Zeng, Wei |
| author_facet | Zhang, Yiwen Hao, Jianing Wang, Zhan Sheng, Hongling Zeng, Wei |
| contents | Video story interaction enables viewers to engage with and explore narrative content for personalized experiences. However, existing methods are limited to user selection, specially designed narratives, and lack customization. To address this, we propose an interactive system based on user intent. Our system uses a Vision Language Model (VLM) to enable machines to understand video stories, combining Retrieval-Augmented Generation (RAG) and a Multi-Agent System (MAS) to create evolving characters and scene experiences. It includes three stages: 1) Video story processing, utilizing VLM and prior knowledge to simulate human understanding of stories across three modalities. 2) Multi-space chat, creating growth-oriented characters through MAS interactions based on user queries and story stages. 3) Scene customization, expanding and visualizing various story scenes mentioned in dialogue. Applied to the Harry Potter series, our study shows the system effectively portrays emergent character social behavior and growth, enhancing the interactive experience in the video story world. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_03807 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Facilitating Video Story Interaction with Multi-Agent Collaborative System Zhang, Yiwen Hao, Jianing Wang, Zhan Sheng, Hongling Zeng, Wei Human-Computer Interaction Artificial Intelligence Computer Vision and Pattern Recognition Multiagent Systems Video story interaction enables viewers to engage with and explore narrative content for personalized experiences. However, existing methods are limited to user selection, specially designed narratives, and lack customization. To address this, we propose an interactive system based on user intent. Our system uses a Vision Language Model (VLM) to enable machines to understand video stories, combining Retrieval-Augmented Generation (RAG) and a Multi-Agent System (MAS) to create evolving characters and scene experiences. It includes three stages: 1) Video story processing, utilizing VLM and prior knowledge to simulate human understanding of stories across three modalities. 2) Multi-space chat, creating growth-oriented characters through MAS interactions based on user queries and story stages. 3) Scene customization, expanding and visualizing various story scenes mentioned in dialogue. Applied to the Harry Potter series, our study shows the system effectively portrays emergent character social behavior and growth, enhancing the interactive experience in the video story world. |
| title | Facilitating Video Story Interaction with Multi-Agent Collaborative System |
| topic | Human-Computer Interaction Artificial Intelligence Computer Vision and Pattern Recognition Multiagent Systems |
| url | https://arxiv.org/abs/2505.03807 |