EvalTalker: Learning to Evaluate Real-Portrait-Driven Multi-Subject Talking Humans
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912740565581824 |
|---|---|
| author | Zhou, Yingjie Zhu, Xilei Ren, Siyu Zhao, Ziyi Wang, Ziwen Wen, Farong Zhou, Yu Cao, Jiezhang Min, Xiongkuo Chen, Fengjiao Li, Xiaoyu Cao, Xuezhi Zhai, Guangtao Liu, Xiaohong |
| author_facet | Zhou, Yingjie Zhu, Xilei Ren, Siyu Zhao, Ziyi Wang, Ziwen Wen, Farong Zhou, Yu Cao, Jiezhang Min, Xiongkuo Chen, Fengjiao Li, Xiaoyu Cao, Xuezhi Zhai, Guangtao Liu, Xiaohong |
| contents | Speech-driven Talking Human (TH) generation, commonly known as "Talker," currently faces limitations in multi-subject driving capabilities. Extending this paradigm to "Multi-Talker," capable of animating multiple subjects simultaneously, introduces richer interactivity and stronger immersion in audiovisual communication. However, current Multi-Talkers still exhibit noticeable quality degradation caused by technical limitations, resulting in suboptimal user experiences. To address this challenge, we construct THQA-MT, the first large-scale Multi-Talker-generated Talking Human Quality Assessment dataset, consisting of 5,492 Multi-Talker-generated THs (MTHs) from 15 representative Multi-Talkers using 400 real portraits collected online. Through subjective experiments, we analyze perceptual discrepancies among different Multi-Talkers and identify 12 common types of distortion. Furthermore, we introduce EvalTalker, a novel TH quality assessment framework. This framework possesses the ability to perceive global quality, human characteristics, and identity consistency, while integrating Qwen-Sync to perceive multimodal synchrony. Experimental results demonstrate that EvalTalker achieves superior correlation with subjective scores, providing a robust foundation for future research on high-quality Multi-Talker generation and evaluation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_01340 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | EvalTalker: Learning to Evaluate Real-Portrait-Driven Multi-Subject Talking Humans Zhou, Yingjie Zhu, Xilei Ren, Siyu Zhao, Ziyi Wang, Ziwen Wen, Farong Zhou, Yu Cao, Jiezhang Min, Xiongkuo Chen, Fengjiao Li, Xiaoyu Cao, Xuezhi Zhai, Guangtao Liu, Xiaohong Computer Vision and Pattern Recognition Speech-driven Talking Human (TH) generation, commonly known as "Talker," currently faces limitations in multi-subject driving capabilities. Extending this paradigm to "Multi-Talker," capable of animating multiple subjects simultaneously, introduces richer interactivity and stronger immersion in audiovisual communication. However, current Multi-Talkers still exhibit noticeable quality degradation caused by technical limitations, resulting in suboptimal user experiences. To address this challenge, we construct THQA-MT, the first large-scale Multi-Talker-generated Talking Human Quality Assessment dataset, consisting of 5,492 Multi-Talker-generated THs (MTHs) from 15 representative Multi-Talkers using 400 real portraits collected online. Through subjective experiments, we analyze perceptual discrepancies among different Multi-Talkers and identify 12 common types of distortion. Furthermore, we introduce EvalTalker, a novel TH quality assessment framework. This framework possesses the ability to perceive global quality, human characteristics, and identity consistency, while integrating Qwen-Sync to perceive multimodal synchrony. Experimental results demonstrate that EvalTalker achieves superior correlation with subjective scores, providing a robust foundation for future research on high-quality Multi-Talker generation and evaluation. |
| title | EvalTalker: Learning to Evaluate Real-Portrait-Driven Multi-Subject Talking Humans |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.01340 |