A Study on Domain Generalization for Failure Detection through Human Reactions in HRI
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917610056056832 |
|---|---|
| author | Parreira, Maria Teresa Lingaraju, Sukruth Gowdru Ramirez-Aristizabal, Adolfo Saha, Manaswi Kuniavsky, Michael Ju, Wendy |
| author_facet | Parreira, Maria Teresa Lingaraju, Sukruth Gowdru Ramirez-Aristizabal, Adolfo Saha, Manaswi Kuniavsky, Michael Ju, Wendy |
| contents | Machine learning models are commonly tested in-distribution (same dataset); performance almost always drops in out-of-distribution settings. For HRI research, the goal is often to develop generalized models. This makes domain generalization - retaining performance in different settings - a critical issue. In this study, we present a concise analysis of domain generalization in failure detection models trained on human facial expressions. Using two distinct datasets of humans reacting to videos where error occurs, one from a controlled lab setting and another collected online, we trained deep learning models on each dataset. When testing these models on the alternate dataset, we observed a significant performance drop. We reflect on the causes for the observed model behavior and leave recommendations. This work emphasizes the need for HRI research focusing on improving model robustness and real-life applicability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_06315 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Study on Domain Generalization for Failure Detection through Human Reactions in HRI Parreira, Maria Teresa Lingaraju, Sukruth Gowdru Ramirez-Aristizabal, Adolfo Saha, Manaswi Kuniavsky, Michael Ju, Wendy Robotics Human-Computer Interaction Machine Learning Machine learning models are commonly tested in-distribution (same dataset); performance almost always drops in out-of-distribution settings. For HRI research, the goal is often to develop generalized models. This makes domain generalization - retaining performance in different settings - a critical issue. In this study, we present a concise analysis of domain generalization in failure detection models trained on human facial expressions. Using two distinct datasets of humans reacting to videos where error occurs, one from a controlled lab setting and another collected online, we trained deep learning models on each dataset. When testing these models on the alternate dataset, we observed a significant performance drop. We reflect on the causes for the observed model behavior and leave recommendations. This work emphasizes the need for HRI research focusing on improving model robustness and real-life applicability. |
| title | A Study on Domain Generalization for Failure Detection through Human Reactions in HRI |
| topic | Robotics Human-Computer Interaction Machine Learning |
| url | https://arxiv.org/abs/2403.06315 |