Exploring Emotion Expression Recognition in Older Adults Interacting with a Virtual Coach
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912087097212928 |
|---|---|
| author | Palmero, Cristina deVelasco, Mikel Hmani, Mohamed Amine Mtibaa, Aymen Letaifa, Leila Ben Buch-Cardona, Pau Justo, Raquel Amorese, Terry González-Fraile, Eduardo Fernández-Ruanova, Begoña Tenorio-Laranga, Jofre Johansen, Anna Torp da Silva, Micaela Rodrigues Martinussen, Liva Jenny Korsnes, Maria Stylianou Cordasco, Gennaro Esposito, Anna El-Yacoubi, Mounim A. Petrovska-Delacrétaz, Dijana Torres, M. Inés Escalera, Sergio |
| author_facet | Palmero, Cristina deVelasco, Mikel Hmani, Mohamed Amine Mtibaa, Aymen Letaifa, Leila Ben Buch-Cardona, Pau Justo, Raquel Amorese, Terry González-Fraile, Eduardo Fernández-Ruanova, Begoña Tenorio-Laranga, Jofre Johansen, Anna Torp da Silva, Micaela Rodrigues Martinussen, Liva Jenny Korsnes, Maria Stylianou Cordasco, Gennaro Esposito, Anna El-Yacoubi, Mounim A. Petrovska-Delacrétaz, Dijana Torres, M. Inés Escalera, Sergio |
| contents | The EMPATHIC project aimed to design an emotionally expressive virtual coach capable of engaging healthy seniors to improve well-being and promote independent aging. One of the core aspects of the system is its human sensing capabilities, allowing for the perception of emotional states to provide a personalized experience. This paper outlines the development of the emotion expression recognition module of the virtual coach, encompassing data collection, annotation design, and a first methodological approach, all tailored to the project requirements. With the latter, we investigate the role of various modalities, individually and combined, for discrete emotion expression recognition in this context: speech from audio, and facial expressions, gaze, and head dynamics from video. The collected corpus includes users from Spain, France, and Norway, and was annotated separately for the audio and video channels with distinct emotional labels, allowing for a performance comparison across cultures and label types. Results confirm the informative power of the modalities studied for the emotional categories considered, with multimodal methods generally outperforming others (around 68% accuracy with audio labels and 72-74% with video labels). The findings are expected to contribute to the limited literature on emotion recognition applied to older adults in conversational human-machine interaction. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_05567 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Exploring Emotion Expression Recognition in Older Adults Interacting with a Virtual Coach Palmero, Cristina deVelasco, Mikel Hmani, Mohamed Amine Mtibaa, Aymen Letaifa, Leila Ben Buch-Cardona, Pau Justo, Raquel Amorese, Terry González-Fraile, Eduardo Fernández-Ruanova, Begoña Tenorio-Laranga, Jofre Johansen, Anna Torp da Silva, Micaela Rodrigues Martinussen, Liva Jenny Korsnes, Maria Stylianou Cordasco, Gennaro Esposito, Anna El-Yacoubi, Mounim A. Petrovska-Delacrétaz, Dijana Torres, M. Inés Escalera, Sergio Computer Vision and Pattern Recognition Human-Computer Interaction Machine Learning The EMPATHIC project aimed to design an emotionally expressive virtual coach capable of engaging healthy seniors to improve well-being and promote independent aging. One of the core aspects of the system is its human sensing capabilities, allowing for the perception of emotional states to provide a personalized experience. This paper outlines the development of the emotion expression recognition module of the virtual coach, encompassing data collection, annotation design, and a first methodological approach, all tailored to the project requirements. With the latter, we investigate the role of various modalities, individually and combined, for discrete emotion expression recognition in this context: speech from audio, and facial expressions, gaze, and head dynamics from video. The collected corpus includes users from Spain, France, and Norway, and was annotated separately for the audio and video channels with distinct emotional labels, allowing for a performance comparison across cultures and label types. Results confirm the informative power of the modalities studied for the emotional categories considered, with multimodal methods generally outperforming others (around 68% accuracy with audio labels and 72-74% with video labels). The findings are expected to contribute to the limited literature on emotion recognition applied to older adults in conversational human-machine interaction. |
| title | Exploring Emotion Expression Recognition in Older Adults Interacting with a Virtual Coach |
| topic | Computer Vision and Pattern Recognition Human-Computer Interaction Machine Learning |
| url | https://arxiv.org/abs/2311.05567 |