"Why the face?": Exploring Robot Error Detection Using Instrumented Bystander Reactions
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908681221701632 |
|---|---|
| author | Parreira, Maria Teresa Zhang, Ruidong Lingaraju, Sukruth Gowdru Bremers, Alexandra Fang, Xuanyu Ramirez-Aristizabal, Adolfo Saha, Manaswi Kuniavsky, Michael Zhang, Cheng Ju, Wendy |
| author_facet | Parreira, Maria Teresa Zhang, Ruidong Lingaraju, Sukruth Gowdru Bremers, Alexandra Fang, Xuanyu Ramirez-Aristizabal, Adolfo Saha, Manaswi Kuniavsky, Michael Zhang, Cheng Ju, Wendy |
| contents | How do humans recognize and rectify social missteps? We achieve social competence by looking around at our peers, decoding subtle cues from bystanders - a raised eyebrow, a laugh - to evaluate the environment and our actions. Robots, however, struggle to perceive and make use of these nuanced reactions. By employing a novel neck-mounted device that records facial expressions from the chin region, we explore the potential of previously untapped data to capture and interpret human responses to robot error. First, we develop NeckNet-18, a 3D facial reconstruction model to map the reactions captured through the chin camera onto facial points and head motion. We then use these facial responses to develop a robot error detection model which outperforms standard methodologies such as using OpenFace or video data, generalizing well especially for within-participant data. Through this work, we argue for expanding human-in-the-loop robot sensing, fostering more seamless integration of robots into diverse human environments, pushing the boundaries of social cue detection and opening new avenues for adaptable robotics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_00262 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | "Why the face?": Exploring Robot Error Detection Using Instrumented Bystander Reactions Parreira, Maria Teresa Zhang, Ruidong Lingaraju, Sukruth Gowdru Bremers, Alexandra Fang, Xuanyu Ramirez-Aristizabal, Adolfo Saha, Manaswi Kuniavsky, Michael Zhang, Cheng Ju, Wendy Robotics Human-Computer Interaction How do humans recognize and rectify social missteps? We achieve social competence by looking around at our peers, decoding subtle cues from bystanders - a raised eyebrow, a laugh - to evaluate the environment and our actions. Robots, however, struggle to perceive and make use of these nuanced reactions. By employing a novel neck-mounted device that records facial expressions from the chin region, we explore the potential of previously untapped data to capture and interpret human responses to robot error. First, we develop NeckNet-18, a 3D facial reconstruction model to map the reactions captured through the chin camera onto facial points and head motion. We then use these facial responses to develop a robot error detection model which outperforms standard methodologies such as using OpenFace or video data, generalizing well especially for within-participant data. Through this work, we argue for expanding human-in-the-loop robot sensing, fostering more seamless integration of robots into diverse human environments, pushing the boundaries of social cue detection and opening new avenues for adaptable robotics. |
| title | "Why the face?": Exploring Robot Error Detection Using Instrumented Bystander Reactions |
| topic | Robotics Human-Computer Interaction |
| url | https://arxiv.org/abs/2512.00262 |