MVP: Multimodal Emotion Recognition based on Video and Physiological Signals
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929660853485568 |
|---|---|
| author | Strizhkova, Valeriya Kachmar, Hadi Chaptoukaev, Hava Kalandadze, Raphael Kukhilava, Natia Tsmindashvili, Tatia Abo-Alzahab, Nibras Zuluaga, Maria A. Balazia, Michal Dantcheva, Antitza Brémond, François Ferrari, Laura |
| author_facet | Strizhkova, Valeriya Kachmar, Hadi Chaptoukaev, Hava Kalandadze, Raphael Kukhilava, Natia Tsmindashvili, Tatia Abo-Alzahab, Nibras Zuluaga, Maria A. Balazia, Michal Dantcheva, Antitza Brémond, François Ferrari, Laura |
| contents | Human emotions entail a complex set of behavioral, physiological and cognitive changes. Current state-of-the-art models fuse the behavioral and physiological components using classic machine learning, rather than recent deep learning techniques. We propose to fill this gap, designing the Multimodal for Video and Physio (MVP) architecture, streamlined to fuse video and physiological signals. Differently then others approaches, MVP exploits the benefits of attention to enable the use of long input sequences (1-2 minutes). We have studied video and physiological backbones for inputting long sequences and evaluated our method with respect to the state-of-the-art. Our results show that MVP outperforms former methods for emotion recognition based on facial videos, EDA, and ECG/PPG. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_03103 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MVP: Multimodal Emotion Recognition based on Video and Physiological Signals Strizhkova, Valeriya Kachmar, Hadi Chaptoukaev, Hava Kalandadze, Raphael Kukhilava, Natia Tsmindashvili, Tatia Abo-Alzahab, Nibras Zuluaga, Maria A. Balazia, Michal Dantcheva, Antitza Brémond, François Ferrari, Laura Computer Vision and Pattern Recognition 68T05, 68T10 I.5 Human emotions entail a complex set of behavioral, physiological and cognitive changes. Current state-of-the-art models fuse the behavioral and physiological components using classic machine learning, rather than recent deep learning techniques. We propose to fill this gap, designing the Multimodal for Video and Physio (MVP) architecture, streamlined to fuse video and physiological signals. Differently then others approaches, MVP exploits the benefits of attention to enable the use of long input sequences (1-2 minutes). We have studied video and physiological backbones for inputting long sequences and evaluated our method with respect to the state-of-the-art. Our results show that MVP outperforms former methods for emotion recognition based on facial videos, EDA, and ECG/PPG. |
| title | MVP: Multimodal Emotion Recognition based on Video and Physiological Signals |
| topic | Computer Vision and Pattern Recognition 68T05, 68T10 I.5 |
| url | https://arxiv.org/abs/2501.03103 |