StoryNavi: On-Demand Narrative-Driven Reconstruction of Video Play With Generative AI
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910632652046336 |
|---|---|
| author | Xu, Alston Lantian Ma, Tianwei Liu, Tianmeng Liu, Can Cassinelli, Alvaro |
| author_facet | Xu, Alston Lantian Ma, Tianwei Liu, Tianmeng Liu, Can Cassinelli, Alvaro |
| contents | Manually navigating lengthy videos to seek information or answer questions can be a tedious and time-consuming task for users. We introduce StoryNavi, a novel system powered by VLLMs for generating customised video play experiences by retrieving materials from original videos. It directly answers users' query by constructing non-linear sequence with identified relevant clips to form a cohesive narrative. StoryNavi offers two modes of playback of the constructed video plays: 1) video-centric, which plays original audio and skips irrelevant segments, and 2) narrative-centric, narration guides the experience, and the original audio is muted. Our technical evaluation showed adequate retrieval performance compared to human retrieval. Our user evaluation shows that maintaining narrative coherence significantly enhances user engagement when viewing disjointed video segments. However, factors like video genre, content, and the query itself may lead to varying user preferences for the playback mode. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_03207 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | StoryNavi: On-Demand Narrative-Driven Reconstruction of Video Play With Generative AI Xu, Alston Lantian Ma, Tianwei Liu, Tianmeng Liu, Can Cassinelli, Alvaro Human-Computer Interaction Manually navigating lengthy videos to seek information or answer questions can be a tedious and time-consuming task for users. We introduce StoryNavi, a novel system powered by VLLMs for generating customised video play experiences by retrieving materials from original videos. It directly answers users' query by constructing non-linear sequence with identified relevant clips to form a cohesive narrative. StoryNavi offers two modes of playback of the constructed video plays: 1) video-centric, which plays original audio and skips irrelevant segments, and 2) narrative-centric, narration guides the experience, and the original audio is muted. Our technical evaluation showed adequate retrieval performance compared to human retrieval. Our user evaluation shows that maintaining narrative coherence significantly enhances user engagement when viewing disjointed video segments. However, factors like video genre, content, and the query itself may lead to varying user preferences for the playback mode. |
| title | StoryNavi: On-Demand Narrative-Driven Reconstruction of Video Play With Generative AI |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2410.03207 |