Affect Decoding in Phonated and Silent Speech Production from Surface EMG
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911530241490944 |
|---|---|
| author | Pistrosch, Simon Avramidis, Kleanthis Ren, Zhao Feng, Tiantian Lee, Jihwan Gonzalez-Machorro, Monica Batliner, Anton Schultz, Tanja Narayanan, Shrikanth Schuller, Björn W. |
| author_facet | Pistrosch, Simon Avramidis, Kleanthis Ren, Zhao Feng, Tiantian Lee, Jihwan Gonzalez-Machorro, Monica Batliner, Anton Schultz, Tanja Narayanan, Shrikanth Schuller, Björn W. |
| contents | The expression of affect is integral to spoken communication, yet, its link to underlying articulatory execution remains unclear. Measures of articulatory muscle activity such as EMG could reveal how speech production is modulated by emotion alongside acoustic speech analyses. We investigate affect decoding from facial and neck surface electromyography (sEMG) during phonated and silent speech production. For this purpose, we introduce a dataset comprising 2,780 utterances from 12 participants across 3 tasks, on which we evaluate both intra- and inter-subject decoding using a range of features and model embeddings. Our results reveal that EMG representations reliably discriminate frustration with up to 0.845 AUC, and generalize well across articulation modes. Our ablation study further demonstrates that affective signatures are embedded in facial motor activity and persist in the absence of phonation, highlighting the potential of EMG sensing for affect-aware silent speech interfaces. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_11715 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Affect Decoding in Phonated and Silent Speech Production from Surface EMG Pistrosch, Simon Avramidis, Kleanthis Ren, Zhao Feng, Tiantian Lee, Jihwan Gonzalez-Machorro, Monica Batliner, Anton Schultz, Tanja Narayanan, Shrikanth Schuller, Björn W. Audio and Speech Processing Artificial Intelligence Sound The expression of affect is integral to spoken communication, yet, its link to underlying articulatory execution remains unclear. Measures of articulatory muscle activity such as EMG could reveal how speech production is modulated by emotion alongside acoustic speech analyses. We investigate affect decoding from facial and neck surface electromyography (sEMG) during phonated and silent speech production. For this purpose, we introduce a dataset comprising 2,780 utterances from 12 participants across 3 tasks, on which we evaluate both intra- and inter-subject decoding using a range of features and model embeddings. Our results reveal that EMG representations reliably discriminate frustration with up to 0.845 AUC, and generalize well across articulation modes. Our ablation study further demonstrates that affective signatures are embedded in facial motor activity and persist in the absence of phonation, highlighting the potential of EMG sensing for affect-aware silent speech interfaces. |
| title | Affect Decoding in Phonated and Silent Speech Production from Surface EMG |
| topic | Audio and Speech Processing Artificial Intelligence Sound |
| url | https://arxiv.org/abs/2603.11715 |