TalkVid: A Large-Scale Diversified Dataset for Audio-Driven Talking Head Synthesis
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915451251982336 |
|---|---|
| author | Chen, Shunian Huang, Hejin Liu, Yexin Ye, Zihan Chen, Pengcheng Zhu, Chenghao Guan, Michael Wang, Rongsheng Chen, Junying Li, Guanbin Lim, Ser-Nam Yang, Harry Wang, Benyou |
| author_facet | Chen, Shunian Huang, Hejin Liu, Yexin Ye, Zihan Chen, Pengcheng Zhu, Chenghao Guan, Michael Wang, Rongsheng Chen, Junying Li, Guanbin Lim, Ser-Nam Yang, Harry Wang, Benyou |
| contents | Audio-driven talking head synthesis has achieved remarkable photorealism, yet state-of-the-art (SOTA) models exhibit a critical failure: they lack generalization to the full spectrum of human diversity in ethnicity, language, and age groups. We argue that this generalization gap is a direct symptom of limitations in existing training data, which lack the necessary scale, quality, and diversity. To address this challenge, we introduce TalkVid, a new large-scale, high-quality, and diverse dataset containing 1244 hours of video from 7729 unique speakers. TalkVid is curated through a principled, multi-stage automated pipeline that rigorously filters for motion stability, aesthetic quality, and facial detail, and is validated against human judgments to ensure its reliability. Furthermore, we construct and release TalkVid-Bench, a stratified evaluation set of 500 clips meticulously balanced across key demographic and linguistic axes. Our experiments demonstrate that a model trained on TalkVid outperforms counterparts trained on previous datasets, exhibiting superior cross-dataset generalization. Crucially, our analysis on TalkVid-Bench reveals performance disparities across subgroups that are obscured by traditional aggregate metrics, underscoring its necessity for future research. Code and data can be found in https://github.com/FreedomIntelligence/TalkVid |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_13618 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TalkVid: A Large-Scale Diversified Dataset for Audio-Driven Talking Head Synthesis Chen, Shunian Huang, Hejin Liu, Yexin Ye, Zihan Chen, Pengcheng Zhu, Chenghao Guan, Michael Wang, Rongsheng Chen, Junying Li, Guanbin Lim, Ser-Nam Yang, Harry Wang, Benyou Computer Vision and Pattern Recognition Audio-driven talking head synthesis has achieved remarkable photorealism, yet state-of-the-art (SOTA) models exhibit a critical failure: they lack generalization to the full spectrum of human diversity in ethnicity, language, and age groups. We argue that this generalization gap is a direct symptom of limitations in existing training data, which lack the necessary scale, quality, and diversity. To address this challenge, we introduce TalkVid, a new large-scale, high-quality, and diverse dataset containing 1244 hours of video from 7729 unique speakers. TalkVid is curated through a principled, multi-stage automated pipeline that rigorously filters for motion stability, aesthetic quality, and facial detail, and is validated against human judgments to ensure its reliability. Furthermore, we construct and release TalkVid-Bench, a stratified evaluation set of 500 clips meticulously balanced across key demographic and linguistic axes. Our experiments demonstrate that a model trained on TalkVid outperforms counterparts trained on previous datasets, exhibiting superior cross-dataset generalization. Crucially, our analysis on TalkVid-Bench reveals performance disparities across subgroups that are obscured by traditional aggregate metrics, underscoring its necessity for future research. Code and data can be found in https://github.com/FreedomIntelligence/TalkVid |
| title | TalkVid: A Large-Scale Diversified Dataset for Audio-Driven Talking Head Synthesis |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2508.13618 |