Multimodal Emotion Recognition and Sentiment Analysis in Multi-Party Conversation Contexts
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915189558870016 |
|---|---|
| author | Farhadipour, Aref Ranjbar, Hossein Chapariniya, Masoumeh Vukovic, Teodora Ebling, Sarah Dellwo, Volker |
| author_facet | Farhadipour, Aref Ranjbar, Hossein Chapariniya, Masoumeh Vukovic, Teodora Ebling, Sarah Dellwo, Volker |
| contents | Emotion recognition and sentiment analysis are pivotal tasks in speech and language processing, particularly in real-world scenarios involving multi-party, conversational data. This paper presents a multimodal approach to tackle these challenges on a well-known dataset. We propose a system that integrates four key modalities/channels using pre-trained models: RoBERTa for text, Wav2Vec2 for speech, a proposed FacialNet for facial expressions, and a CNN+Transformer architecture trained from scratch for video analysis. Feature embeddings from each modality are concatenated to form a multimodal vector, which is then used to predict emotion and sentiment labels. The multimodal system demonstrates superior performance compared to unimodal approaches, achieving an accuracy of 66.36% for emotion recognition and 72.15% for sentiment analysis. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_06805 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Multimodal Emotion Recognition and Sentiment Analysis in Multi-Party Conversation Contexts Farhadipour, Aref Ranjbar, Hossein Chapariniya, Masoumeh Vukovic, Teodora Ebling, Sarah Dellwo, Volker Computer Vision and Pattern Recognition Sound Audio and Speech Processing Emotion recognition and sentiment analysis are pivotal tasks in speech and language processing, particularly in real-world scenarios involving multi-party, conversational data. This paper presents a multimodal approach to tackle these challenges on a well-known dataset. We propose a system that integrates four key modalities/channels using pre-trained models: RoBERTa for text, Wav2Vec2 for speech, a proposed FacialNet for facial expressions, and a CNN+Transformer architecture trained from scratch for video analysis. Feature embeddings from each modality are concatenated to form a multimodal vector, which is then used to predict emotion and sentiment labels. The multimodal system demonstrates superior performance compared to unimodal approaches, achieving an accuracy of 66.36% for emotion recognition and 72.15% for sentiment analysis. |
| title | Multimodal Emotion Recognition and Sentiment Analysis in Multi-Party Conversation Contexts |
| topic | Computer Vision and Pattern Recognition Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2503.06805 |