Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2506.01689 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909632806518784 |
|---|---|
| author | Wang, Shuting Liu, Yunqi Yang, Zixin Hu, Ning Dou, Zhicheng Xiong, Chenyan |
| author_facet | Wang, Shuting Liu, Yunqi Yang, Zixin Hu, Ning Dou, Zhicheng Xiong, Chenyan |
| contents | Querying generative AI models, e.g., large language models (LLMs), has become a prevalent method for information acquisition. However, existing query-answer datasets primarily focus on textual responses, making it challenging to address complex user queries that require visual demonstrations or explanations for better understanding. To bridge this gap, we construct a benchmark, RealVideoQuest, designed to evaluate the abilities of text-to-video (T2V) models in answering real-world, visually grounded queries. It identifies 7.5K real user queries with video response intents from Chatbot-Arena and builds 4.5K high-quality query-video pairs through a multistage video retrieval and refinement process. We further develop a multi-angle evaluation system to assess the quality of generated video answers. Experiments indicate that current T2V models struggle with effectively addressing real user queries, pointing to key challenges and future research opportunities in multimodal AI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_01689 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Respond Beyond Language: A Benchmark for Video Generation in Response to Realistic User Intents Wang, Shuting Liu, Yunqi Yang, Zixin Hu, Ning Dou, Zhicheng Xiong, Chenyan Artificial Intelligence Computation and Language Querying generative AI models, e.g., large language models (LLMs), has become a prevalent method for information acquisition. However, existing query-answer datasets primarily focus on textual responses, making it challenging to address complex user queries that require visual demonstrations or explanations for better understanding. To bridge this gap, we construct a benchmark, RealVideoQuest, designed to evaluate the abilities of text-to-video (T2V) models in answering real-world, visually grounded queries. It identifies 7.5K real user queries with video response intents from Chatbot-Arena and builds 4.5K high-quality query-video pairs through a multistage video retrieval and refinement process. We further develop a multi-angle evaluation system to assess the quality of generated video answers. Experiments indicate that current T2V models struggle with effectively addressing real user queries, pointing to key challenges and future research opportunities in multimodal AI. |
| title | Respond Beyond Language: A Benchmark for Video Generation in Response to Realistic User Intents |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2506.01689 |