User Experience Estimation in Human-Robot Interaction Via Multi-Instance Learning of Multimodal Social Signals
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916873542565888 |
|---|---|
| author | Miyoshi, Ryo Okafuji, Yuki Iwamoto, Takuya Nakanishi, Junya Baba, Jun |
| author_facet | Miyoshi, Ryo Okafuji, Yuki Iwamoto, Takuya Nakanishi, Junya Baba, Jun |
| contents | In recent years, the demand for social robots has grown, requiring them to adapt their behaviors based on users' states. Accurately assessing user experience (UX) in human-robot interaction (HRI) is crucial for achieving this adaptability. UX is a multi-faceted measure encompassing aspects such as sentiment and engagement, yet existing methods often focus on these individually. This study proposes a UX estimation method for HRI by leveraging multimodal social signals. We construct a UX dataset and develop a Transformer-based model that utilizes facial expressions and voice for estimation. Unlike conventional models that rely on momentary observations, our approach captures both short- and long-term interaction patterns using a multi-instance learning framework. This enables the model to capture temporal dynamics in UX, providing a more holistic representation. Experimental results demonstrate that our method outperforms third-party human evaluators in UX estimation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_23544 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | User Experience Estimation in Human-Robot Interaction Via Multi-Instance Learning of Multimodal Social Signals Miyoshi, Ryo Okafuji, Yuki Iwamoto, Takuya Nakanishi, Junya Baba, Jun Robotics Computer Vision and Pattern Recognition Human-Computer Interaction In recent years, the demand for social robots has grown, requiring them to adapt their behaviors based on users' states. Accurately assessing user experience (UX) in human-robot interaction (HRI) is crucial for achieving this adaptability. UX is a multi-faceted measure encompassing aspects such as sentiment and engagement, yet existing methods often focus on these individually. This study proposes a UX estimation method for HRI by leveraging multimodal social signals. We construct a UX dataset and develop a Transformer-based model that utilizes facial expressions and voice for estimation. Unlike conventional models that rely on momentary observations, our approach captures both short- and long-term interaction patterns using a multi-instance learning framework. This enables the model to capture temporal dynamics in UX, providing a more holistic representation. Experimental results demonstrate that our method outperforms third-party human evaluators in UX estimation. |
| title | User Experience Estimation in Human-Robot Interaction Via Multi-Instance Learning of Multimodal Social Signals |
| topic | Robotics Computer Vision and Pattern Recognition Human-Computer Interaction |
| url | https://arxiv.org/abs/2507.23544 |