MERCI: Multimodal Emotional and peRsonal Conversational Interactions Dataset
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912279060021248 |
|---|---|
| author | Althubyani, Mohammed Meng, Zhijin Xie, Shengyuan Seung, Cha Razzak, Imran Sandoval, Eduardo B. Kocaballi, Baki Cruz, Francisco |
| author_facet | Althubyani, Mohammed Meng, Zhijin Xie, Shengyuan Seung, Cha Razzak, Imran Sandoval, Eduardo B. Kocaballi, Baki Cruz, Francisco |
| contents | The integration of conversational agents into our daily lives has become increasingly common, yet many of these agents cannot engage in deep interactions with humans. Despite this, there is a noticeable shortage of datasets that capture multimodal information from human-robot interaction dialogues. To address this gap, we have recorded a novel multimodal dataset (MERCI) that encompasses rich embodied interaction data. The process involved asking participants to complete a questionnaire and gathering their profiles on ten topics, such as hobbies and favorite music. Subsequently, we initiated conversations between the robot and the participants, leveraging GPT-4 to generate contextually appropriate responses based on the participant's profile and emotional state, as determined by facial expression recognition and sentiment analysis. Automatic and user evaluations were conducted to assess the overall quality of the collected data. The results of both evaluations indicated a high level of naturalness, engagement, fluency, consistency, and relevance in the conversation, as well as the robot's ability to provide empathetic responses. It is worth noting that the dataset is derived from genuine interactions with the robot, involving participants who provided personal information and conveyed actual emotions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_04908 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MERCI: Multimodal Emotional and peRsonal Conversational Interactions Dataset Althubyani, Mohammed Meng, Zhijin Xie, Shengyuan Seung, Cha Razzak, Imran Sandoval, Eduardo B. Kocaballi, Baki Cruz, Francisco Human-Computer Interaction Emerging Technologies Robotics K.4.0 The integration of conversational agents into our daily lives has become increasingly common, yet many of these agents cannot engage in deep interactions with humans. Despite this, there is a noticeable shortage of datasets that capture multimodal information from human-robot interaction dialogues. To address this gap, we have recorded a novel multimodal dataset (MERCI) that encompasses rich embodied interaction data. The process involved asking participants to complete a questionnaire and gathering their profiles on ten topics, such as hobbies and favorite music. Subsequently, we initiated conversations between the robot and the participants, leveraging GPT-4 to generate contextually appropriate responses based on the participant's profile and emotional state, as determined by facial expression recognition and sentiment analysis. Automatic and user evaluations were conducted to assess the overall quality of the collected data. The results of both evaluations indicated a high level of naturalness, engagement, fluency, consistency, and relevance in the conversation, as well as the robot's ability to provide empathetic responses. It is worth noting that the dataset is derived from genuine interactions with the robot, involving participants who provided personal information and conveyed actual emotions. |
| title | MERCI: Multimodal Emotional and peRsonal Conversational Interactions Dataset |
| topic | Human-Computer Interaction Emerging Technologies Robotics K.4.0 |
| url | https://arxiv.org/abs/2412.04908 |