MTAVG-Bench 2.0: Diagnosing Failure Modes of Cinematic Expressiveness in Multi-Talker Audio-Video Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913166842134528 |
|---|---|
| author | Li, Haitian Zhou, Yanghao Huang, Heyan Chen, Liangji Cheng, YiMing Liu, Xu Jin, Dian Xu, Jiajun Liao, Jingyun Lan, Tian Zhou, Ziqin Liu, Yueying Bai, Yu Yuan, Changsen Zhou, Jinxing Mao, Xian-Ling Chen, Xuefeng Feng, Yousheng |
| author_facet | Li, Haitian Zhou, Yanghao Huang, Heyan Chen, Liangji Cheng, YiMing Liu, Xu Jin, Dian Xu, Jiajun Liao, Jingyun Lan, Tian Zhou, Ziqin Liu, Yueying Bai, Yu Yuan, Changsen Zhou, Jinxing Mao, Xian-Ling Chen, Xuefeng Feng, Yousheng |
| contents | In recent years, Multi-Talker Audio-Video Generation (MTAVG) models have shown promising performance on fundamental metrics such as lip-sync and audio-visual alignment. However, these metrics remain insufficient for assessing cinematic expressiveness in scene-level generation. In multi-character scenes, generation models must go beyond audio-visual realism to convey coherent character performance and other higher-level cinematic qualities. To fill this gap, we introduce MTAVG-Bench 2.0, a benchmark for diagnosing failure modes of cinematic expressiveness in multi-talker audio-video generation. Unlike prior settings that mainly focus on the quality of basic multi-turn dialogue, MTAVG-Bench 2.0 targets short-drama and scene-level generation, and establishes a high-level failure taxonomy spanning acting, narrative, atmosphere, and audio-visual language. Based on this taxonomy, we construct more than 10,000 question-answering evaluation instances, together with subsets for short-drama-level assessment and temporal localization of failure modes, to systematically evaluate the ability of omni large language models to diagnose high-level audio-visual failures. Experimental results show that commercial omni models such as Gemini substantially outperform other evaluators, yet even the strongest models continue to struggle with complex failures in our benchmark. These results demonstrate that MTAVG-Bench 2.0 provides a systematic benchmark for failure diagnosis in cinematic multi-talker audio-video generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_28035 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | MTAVG-Bench 2.0: Diagnosing Failure Modes of Cinematic Expressiveness in Multi-Talker Audio-Video Generation Li, Haitian Zhou, Yanghao Huang, Heyan Chen, Liangji Cheng, YiMing Liu, Xu Jin, Dian Xu, Jiajun Liao, Jingyun Lan, Tian Zhou, Ziqin Liu, Yueying Bai, Yu Yuan, Changsen Zhou, Jinxing Mao, Xian-Ling Chen, Xuefeng Feng, Yousheng Artificial Intelligence Multimedia Sound In recent years, Multi-Talker Audio-Video Generation (MTAVG) models have shown promising performance on fundamental metrics such as lip-sync and audio-visual alignment. However, these metrics remain insufficient for assessing cinematic expressiveness in scene-level generation. In multi-character scenes, generation models must go beyond audio-visual realism to convey coherent character performance and other higher-level cinematic qualities. To fill this gap, we introduce MTAVG-Bench 2.0, a benchmark for diagnosing failure modes of cinematic expressiveness in multi-talker audio-video generation. Unlike prior settings that mainly focus on the quality of basic multi-turn dialogue, MTAVG-Bench 2.0 targets short-drama and scene-level generation, and establishes a high-level failure taxonomy spanning acting, narrative, atmosphere, and audio-visual language. Based on this taxonomy, we construct more than 10,000 question-answering evaluation instances, together with subsets for short-drama-level assessment and temporal localization of failure modes, to systematically evaluate the ability of omni large language models to diagnose high-level audio-visual failures. Experimental results show that commercial omni models such as Gemini substantially outperform other evaluators, yet even the strongest models continue to struggle with complex failures in our benchmark. These results demonstrate that MTAVG-Bench 2.0 provides a systematic benchmark for failure diagnosis in cinematic multi-talker audio-video generation. |
| title | MTAVG-Bench 2.0: Diagnosing Failure Modes of Cinematic Expressiveness in Multi-Talker Audio-Video Generation |
| topic | Artificial Intelligence Multimedia Sound |
| url | https://arxiv.org/abs/2605.28035 |