Reward Learning from Suboptimal Demonstrations with Applications in Surgical Electrocautery
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914968770707456 |
|---|---|
| author | Karimi, Zohre Ho, Shing-Hei Thach, Bao Kuntz, Alan Brown, Daniel S. |
| author_facet | Karimi, Zohre Ho, Shing-Hei Thach, Bao Kuntz, Alan Brown, Daniel S. |
| contents | Automating robotic surgery via learning from demonstration (LfD) techniques is extremely challenging. This is because surgical tasks often involve sequential decision-making processes with complex interactions of physical objects and have low tolerance for mistakes. Prior works assume that all demonstrations are fully observable and optimal, which might not be practical in the real world. This paper introduces a sample-efficient method that learns a robust reward function from a limited amount of ranked suboptimal demonstrations consisting of partial-view point cloud observations. The method then learns a policy by optimizing the learned reward function using reinforcement learning (RL). We show that using a learned reward function to obtain a policy is more robust than pure imitation learning. We apply our approach on a physical surgical electrocautery task and demonstrate that our method can perform well even when the provided demonstrations are suboptimal and the observations are high-dimensional point clouds. Code and videos available here: https://sites.google.com/view/lfdinelectrocautery |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_07185 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Reward Learning from Suboptimal Demonstrations with Applications in Surgical Electrocautery Karimi, Zohre Ho, Shing-Hei Thach, Bao Kuntz, Alan Brown, Daniel S. Robotics Artificial Intelligence Machine Learning Automating robotic surgery via learning from demonstration (LfD) techniques is extremely challenging. This is because surgical tasks often involve sequential decision-making processes with complex interactions of physical objects and have low tolerance for mistakes. Prior works assume that all demonstrations are fully observable and optimal, which might not be practical in the real world. This paper introduces a sample-efficient method that learns a robust reward function from a limited amount of ranked suboptimal demonstrations consisting of partial-view point cloud observations. The method then learns a policy by optimizing the learned reward function using reinforcement learning (RL). We show that using a learned reward function to obtain a policy is more robust than pure imitation learning. We apply our approach on a physical surgical electrocautery task and demonstrate that our method can perform well even when the provided demonstrations are suboptimal and the observations are high-dimensional point clouds. Code and videos available here: https://sites.google.com/view/lfdinelectrocautery |
| title | Reward Learning from Suboptimal Demonstrations with Applications in Surgical Electrocautery |
| topic | Robotics Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2404.07185 |