Learning to Evaluate Autonomous Behaviour in Human-Robot Interaction
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913933206487040 |
|---|---|
| author | Tiezzi, Matteo Apicella, Tommaso Cardenas-Perez, Carlos Fregonese, Giovanni Dafarra, Stefano Morerio, Pietro Pucci, Daniele Del Bue, Alessio |
| author_facet | Tiezzi, Matteo Apicella, Tommaso Cardenas-Perez, Carlos Fregonese, Giovanni Dafarra, Stefano Morerio, Pietro Pucci, Daniele Del Bue, Alessio |
| contents | Evaluating and comparing the performance of autonomous Humanoid Robots is challenging, as success rate metrics are difficult to reproduce and fail to capture the complexity of robot movement trajectories, critical in Human-Robot Interaction and Collaboration (HRIC). To address these challenges, we propose a general evaluation framework that measures the quality of Imitation Learning (IL) methods by focusing on trajectory performance. We devise the Neural Meta Evaluator (NeME), a deep learning model trained to classify actions from robot joint trajectories. NeME serves as a meta-evaluator to compare the performance of robot control policies, enabling policy evaluation without requiring human involvement in the loop. We validate our framework on ergoCub, a humanoid robot, using teleoperation data and comparing IL methods tailored to the available platform. The experimental results indicate that our method is more aligned with the success rate obtained on the robot than baselines, offering a reproducible, systematic, and insightful means for comparing the performance of multimodal imitation learning approaches in complex HRI tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_06404 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Learning to Evaluate Autonomous Behaviour in Human-Robot Interaction Tiezzi, Matteo Apicella, Tommaso Cardenas-Perez, Carlos Fregonese, Giovanni Dafarra, Stefano Morerio, Pietro Pucci, Daniele Del Bue, Alessio Robotics Computer Vision and Pattern Recognition Machine Learning Evaluating and comparing the performance of autonomous Humanoid Robots is challenging, as success rate metrics are difficult to reproduce and fail to capture the complexity of robot movement trajectories, critical in Human-Robot Interaction and Collaboration (HRIC). To address these challenges, we propose a general evaluation framework that measures the quality of Imitation Learning (IL) methods by focusing on trajectory performance. We devise the Neural Meta Evaluator (NeME), a deep learning model trained to classify actions from robot joint trajectories. NeME serves as a meta-evaluator to compare the performance of robot control policies, enabling policy evaluation without requiring human involvement in the loop. We validate our framework on ergoCub, a humanoid robot, using teleoperation data and comparing IL methods tailored to the available platform. The experimental results indicate that our method is more aligned with the success rate obtained on the robot than baselines, offering a reproducible, systematic, and insightful means for comparing the performance of multimodal imitation learning approaches in complex HRI tasks. |
| title | Learning to Evaluate Autonomous Behaviour in Human-Robot Interaction |
| topic | Robotics Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2507.06404 |