TDMM-LM: Bridging Facial Understanding and Animation via Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910057317269504 |
|---|---|
| author | Song, Luchuan Liu, Pinxin Liu, Haiyang Jin, Zhenchao Tang, Yolo Yunlong Xu, Zichong Liang, Susan Bi, Jing Corso, Jason J Xu, Chenliang |
| author_facet | Song, Luchuan Liu, Pinxin Liu, Haiyang Jin, Zhenchao Tang, Yolo Yunlong Xu, Zichong Liang, Susan Bi, Jing Corso, Jason J Xu, Chenliang |
| contents | Text-guided human body animation has advanced rapidly, yet facial animation lags due to the scarcity of well-annotated, text-paired facial corpora. To close this gap, we leverage foundation generative models to synthesize a large, balanced corpus of facial behavior. We design prompts suite covering emotions and head motions, generate about 80 hours of facial videos with multiple generators, and fit per-frame 3D facial parameters, yielding large-scale (prompt and parameter) pairs for training. Building on this dataset, we probe language models for bidirectional competence over facial motion via two complementary tasks: (1) Motion2Language: given a sequence of 3D facial parameters, the model produces natural-language descriptions capturing content, style, and dynamics; and (2) Language2Motion: given a prompt, the model synthesizes the corresponding sequence of 3D facial parameters via quantized motion tokens for downstream animation. Extensive experiments show that in this setting language models can both interpret and synthesize facial motion with strong generalization. To best of our knowledge, this is the first work to cast facial-parameter modeling as a language problem, establishing a unified path for text-conditioned facial animation and motion understanding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_16936 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | TDMM-LM: Bridging Facial Understanding and Animation via Language Models Song, Luchuan Liu, Pinxin Liu, Haiyang Jin, Zhenchao Tang, Yolo Yunlong Xu, Zichong Liang, Susan Bi, Jing Corso, Jason J Xu, Chenliang Computer Vision and Pattern Recognition Artificial Intelligence Text-guided human body animation has advanced rapidly, yet facial animation lags due to the scarcity of well-annotated, text-paired facial corpora. To close this gap, we leverage foundation generative models to synthesize a large, balanced corpus of facial behavior. We design prompts suite covering emotions and head motions, generate about 80 hours of facial videos with multiple generators, and fit per-frame 3D facial parameters, yielding large-scale (prompt and parameter) pairs for training. Building on this dataset, we probe language models for bidirectional competence over facial motion via two complementary tasks: (1) Motion2Language: given a sequence of 3D facial parameters, the model produces natural-language descriptions capturing content, style, and dynamics; and (2) Language2Motion: given a prompt, the model synthesizes the corresponding sequence of 3D facial parameters via quantized motion tokens for downstream animation. Extensive experiments show that in this setting language models can both interpret and synthesize facial motion with strong generalization. To best of our knowledge, this is the first work to cast facial-parameter modeling as a language problem, establishing a unified path for text-conditioned facial animation and motion understanding. |
| title | TDMM-LM: Bridging Facial Understanding and Animation via Language Models |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2603.16936 |