Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.14827 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917350901547008 |
|---|---|
| author | Kang, Zejian Zheng, Kai Fei, Yuanchen Yang, Wentao Zou, Hongyuan Huang, Xiangru |
| author_facet | Kang, Zejian Zheng, Kai Fei, Yuanchen Yang, Wentao Zou, Hongyuan Huang, Xiangru |
| contents | Facial action estimation from a single image is often formulated as predicting or fitting parameters in compact expression spaces, which lack explicit semantic interpretability. However, many practical applications, such as avatar control and human-computer interaction, require interpretable facial actions that correspond to meaningful muscle movements. In this work, we propose SemanticFace, a framework for facial action estimation in the interpretable ARKit blendshape space that reformulates coefficient prediction as structured semantic reasoning. SemanticFace adopts a two-stage semantic distillation paradigm: it first derives structured semantic supervision from ground-truth ARKit coefficients and then distills this knowledge into a multimodal large language model to predict interpretable facial action coefficients from images. Extensive experiments demonstrate that language-aligned semantic supervision improves both coefficient accuracy and perceptual consistency, while enabling strong cross-identity generalization and robustness to large domain shifts, including cartoon faces. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_14827 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SemanticFace: Semantic Facial Action Estimation via Semantic Distillation in Interpretable Space Kang, Zejian Zheng, Kai Fei, Yuanchen Yang, Wentao Zou, Hongyuan Huang, Xiangru Computer Vision and Pattern Recognition Facial action estimation from a single image is often formulated as predicting or fitting parameters in compact expression spaces, which lack explicit semantic interpretability. However, many practical applications, such as avatar control and human-computer interaction, require interpretable facial actions that correspond to meaningful muscle movements. In this work, we propose SemanticFace, a framework for facial action estimation in the interpretable ARKit blendshape space that reformulates coefficient prediction as structured semantic reasoning. SemanticFace adopts a two-stage semantic distillation paradigm: it first derives structured semantic supervision from ground-truth ARKit coefficients and then distills this knowledge into a multimodal large language model to predict interpretable facial action coefficients from images. Extensive experiments demonstrate that language-aligned semantic supervision improves both coefficient accuracy and perceptual consistency, while enabling strong cross-identity generalization and robustness to large domain shifts, including cartoon faces. |
| title | SemanticFace: Semantic Facial Action Estimation via Semantic Distillation in Interpretable Space |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.14827 |