Mind-to-Face: Neural-Driven Photorealistic Avatar Synthesis via EEG Decoding
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909943280435200 |
|---|---|
| author | Xiong, Haolin Fu, Tianwen Prasad, Pratusha Bhuvana Cai, Yunxuan Chen, Haiwei Teng, Wenbin Xiao, Hanyuan Zhao, Yajie |
| author_facet | Xiong, Haolin Fu, Tianwen Prasad, Pratusha Bhuvana Cai, Yunxuan Chen, Haiwei Teng, Wenbin Xiao, Hanyuan Zhao, Yajie |
| contents | Current expressive avatar systems rely heavily on visual cues, failing when faces are occluded or when emotions remain internal. We present Mind-to-Face, the first framework that decodes non-invasive electroencephalogram (EEG) signals directly into high-fidelity facial expressions. We build a dual-modality recording setup to obtain synchronized EEG and multi-view facial video during emotion-eliciting stimuli, enabling precise supervision for neural-to-visual learning. Our model uses a CNN-Transformer encoder to map EEG signals into dense 3D position maps, capable of sampling over 65k vertices, capturing fine-scale geometry and subtle emotional dynamics, and renders them through a modified 3D Gaussian Splatting pipeline for photorealistic, view-consistent results. Through extensive evaluation, we show that EEG alone can reliably predict dynamic, subject-specific facial expressions, including subtle emotional responses, demonstrating that neural signals contain far richer affective and geometric information than previously assumed. Mind-to-Face establishes a new paradigm for neural-driven avatars, enabling personalized, emotion-aware telepresence and cognitive interaction in immersive environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_04313 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Mind-to-Face: Neural-Driven Photorealistic Avatar Synthesis via EEG Decoding Xiong, Haolin Fu, Tianwen Prasad, Pratusha Bhuvana Cai, Yunxuan Chen, Haiwei Teng, Wenbin Xiao, Hanyuan Zhao, Yajie Computer Vision and Pattern Recognition Current expressive avatar systems rely heavily on visual cues, failing when faces are occluded or when emotions remain internal. We present Mind-to-Face, the first framework that decodes non-invasive electroencephalogram (EEG) signals directly into high-fidelity facial expressions. We build a dual-modality recording setup to obtain synchronized EEG and multi-view facial video during emotion-eliciting stimuli, enabling precise supervision for neural-to-visual learning. Our model uses a CNN-Transformer encoder to map EEG signals into dense 3D position maps, capable of sampling over 65k vertices, capturing fine-scale geometry and subtle emotional dynamics, and renders them through a modified 3D Gaussian Splatting pipeline for photorealistic, view-consistent results. Through extensive evaluation, we show that EEG alone can reliably predict dynamic, subject-specific facial expressions, including subtle emotional responses, demonstrating that neural signals contain far richer affective and geometric information than previously assumed. Mind-to-Face establishes a new paradigm for neural-driven avatars, enabling personalized, emotion-aware telepresence and cognitive interaction in immersive environments. |
| title | Mind-to-Face: Neural-Driven Photorealistic Avatar Synthesis via EEG Decoding |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.04313 |