محفوظ في:
| المؤلفون الرئيسيون: | , , , , , , , , , , , , , , |
|---|---|
| التنسيق: | Preprint |
| منشور في: |
2026
|
| الموضوعات: | |
| الوصول للمادة أونلاين: | https://arxiv.org/abs/2602.00607 |
| الوسوم: |
إضافة وسم
لا توجد وسوم, كن أول من يضع وسما على هذه التسجيلة!
|
| _version_ | 1866915971096117248 |
|---|---|
| author | Zhou, Yang-Hao Li, Haitian Lin, Rexar Huang, Heyan Zhou, Jinxing Yuan, Changsen Lan, Tian Zhou, Ziqin Li, Yudong Xu, Jiajun Liao, Jingyun Cheng, Yi-Ming Chen, Xuefeng Mao, Xian-Ling Feng, Yousheng |
| author_facet | Zhou, Yang-Hao Li, Haitian Lin, Rexar Huang, Heyan Zhou, Jinxing Yuan, Changsen Lan, Tian Zhou, Ziqin Li, Yudong Xu, Jiajun Liao, Jingyun Cheng, Yi-Ming Chen, Xuefeng Mao, Xian-Ling Feng, Yousheng |
| contents | Recent advances in text-to-audio-video (T2AV) generation have enabled models to synthesize audio-visual videos with multi-participant dialogues. However, existing evaluation benchmarks remain largely designed for human-recorded videos or single-speaker settings. As a result, structural failures in generated multi-talker dialogue videos, such as identity drift, unnatural turn transitions, and audio-visual misalignment, cannot be effectively diagnosed. To address this issue, we introduce MTAVG-Bench, a failure-driven diagnostic benchmark for multi-talker dialogue-centric audio-video generation. MTAVG-Bench is built via a semi-automatic pipeline, where 1.8k videos are generated using mainstream T2AV models with carefully designed prompts, yielding 2.4k manually annotated QA pairs for fine-grained failure diagnosis. The benchmark evaluates multi-speaker dialogue generation at four levels: audio-visual signal fidelity, temporal attribute consistency, social interaction, and cinematic expression. Built on a hierarchical failure taxonomy and a targeted QA protocol, MTAVG-Bench is primarily designed to evaluate whether proprietary and open-source omni-models can reliably identify failure modes in multi-speaker T2AV outputs. We benchmark 12 proprietary and open-source omni-models on MTAVG-Bench, with Gemini 3 Pro achieving the strongest overall performance, while leading open-source models remain competitive in signal fidelity and consistency. Overall, MTAVG-Bench enables fine-grained failure analysis for rigorous model comparison and targeted video generation refinement. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_00607 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | MTAVG-Bench: A Diagnostic Benchmark for Multi-Talker Dialogue-Centric Audio-Video Generation Zhou, Yang-Hao Li, Haitian Lin, Rexar Huang, Heyan Zhou, Jinxing Yuan, Changsen Lan, Tian Zhou, Ziqin Li, Yudong Xu, Jiajun Liao, Jingyun Cheng, Yi-Ming Chen, Xuefeng Mao, Xian-Ling Feng, Yousheng Multimedia Sound Recent advances in text-to-audio-video (T2AV) generation have enabled models to synthesize audio-visual videos with multi-participant dialogues. However, existing evaluation benchmarks remain largely designed for human-recorded videos or single-speaker settings. As a result, structural failures in generated multi-talker dialogue videos, such as identity drift, unnatural turn transitions, and audio-visual misalignment, cannot be effectively diagnosed. To address this issue, we introduce MTAVG-Bench, a failure-driven diagnostic benchmark for multi-talker dialogue-centric audio-video generation. MTAVG-Bench is built via a semi-automatic pipeline, where 1.8k videos are generated using mainstream T2AV models with carefully designed prompts, yielding 2.4k manually annotated QA pairs for fine-grained failure diagnosis. The benchmark evaluates multi-speaker dialogue generation at four levels: audio-visual signal fidelity, temporal attribute consistency, social interaction, and cinematic expression. Built on a hierarchical failure taxonomy and a targeted QA protocol, MTAVG-Bench is primarily designed to evaluate whether proprietary and open-source omni-models can reliably identify failure modes in multi-speaker T2AV outputs. We benchmark 12 proprietary and open-source omni-models on MTAVG-Bench, with Gemini 3 Pro achieving the strongest overall performance, while leading open-source models remain competitive in signal fidelity and consistency. Overall, MTAVG-Bench enables fine-grained failure analysis for rigorous model comparison and targeted video generation refinement. |
| title | MTAVG-Bench: A Diagnostic Benchmark for Multi-Talker Dialogue-Centric Audio-Video Generation |
| topic | Multimedia Sound |
| url | https://arxiv.org/abs/2602.00607 |