UniTAF: A Modular Framework for Joint Text-to-Speech and Audio-to-Face Modeling
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911480923815936 |
|---|---|
| author | Zhou, Qiangong Tomohiro, Nagasaka |
| author_facet | Zhou, Qiangong Tomohiro, Nagasaka |
| contents | This work considers merging two independent models, TTS and A2F, into a unified model to enable internal feature transfer, thereby improving the consistency between audio and facial expressions generated from text. We also discuss the extension of the emotion control mechanism from TTS to the joint model. This work does not aim to showcase generation quality; instead, from a system design perspective, it validates the feasibility of reusing intermediate representations from TTS for joint modeling of speech and facial expressions, and provides engineering practice references for subsequent speech expression co-design. The project code has been open source at: https://github.com/GoldenFishes/UniTAF |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_15651 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | UniTAF: A Modular Framework for Joint Text-to-Speech and Audio-to-Face Modeling Zhou, Qiangong Tomohiro, Nagasaka Sound Computer Vision and Pattern Recognition Audio and Speech Processing 68T10 I.2.0 This work considers merging two independent models, TTS and A2F, into a unified model to enable internal feature transfer, thereby improving the consistency between audio and facial expressions generated from text. We also discuss the extension of the emotion control mechanism from TTS to the joint model. This work does not aim to showcase generation quality; instead, from a system design perspective, it validates the feasibility of reusing intermediate representations from TTS for joint modeling of speech and facial expressions, and provides engineering practice references for subsequent speech expression co-design. The project code has been open source at: https://github.com/GoldenFishes/UniTAF |
| title | UniTAF: A Modular Framework for Joint Text-to-Speech and Audio-to-Face Modeling |
| topic | Sound Computer Vision and Pattern Recognition Audio and Speech Processing 68T10 I.2.0 |
| url | https://arxiv.org/abs/2602.15651 |