On the Vulnerability of LLM/VLM-Controlled Robotics
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916645433245696 |
|---|---|
| author | Wu, Xiyang Chakraborty, Souradip Xian, Ruiqi Liang, Jing Guan, Tianrui Liu, Fuxiao Sadler, Brian M. Manocha, Dinesh Bedi, Amrit Singh |
| author_facet | Wu, Xiyang Chakraborty, Souradip Xian, Ruiqi Liang, Jing Guan, Tianrui Liu, Fuxiao Sadler, Brian M. Manocha, Dinesh Bedi, Amrit Singh |
| contents | In this work, we highlight vulnerabilities in robotic systems integrating large language models (LLMs) and vision-language models (VLMs) due to input modality sensitivities. While LLM/VLM-controlled robots show impressive performance across various tasks, their reliability under slight input variations remains underexplored yet critical. These models are highly sensitive to instruction or perceptual input changes, which can trigger misalignment issues, leading to execution failures with severe real-world consequences. To study this issue, we analyze the misalignment-induced vulnerabilities within LLM/VLM-controlled robotic systems and present a mathematical formulation for failure modes arising from variations in input modalities. We propose empirical perturbation strategies to expose these vulnerabilities and validate their effectiveness through experiments on multiple robot manipulation tasks. Our results show that simple input perturbations reduce task execution success rates by 22.2% and 14.6% in two representative LLM/VLM-controlled robotic systems. These findings underscore the importance of input modality robustness and motivate further research to ensure the safe and reliable deployment of advanced LLM/VLM-controlled robotic systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_10340 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | On the Vulnerability of LLM/VLM-Controlled Robotics Wu, Xiyang Chakraborty, Souradip Xian, Ruiqi Liang, Jing Guan, Tianrui Liu, Fuxiao Sadler, Brian M. Manocha, Dinesh Bedi, Amrit Singh Robotics Artificial Intelligence In this work, we highlight vulnerabilities in robotic systems integrating large language models (LLMs) and vision-language models (VLMs) due to input modality sensitivities. While LLM/VLM-controlled robots show impressive performance across various tasks, their reliability under slight input variations remains underexplored yet critical. These models are highly sensitive to instruction or perceptual input changes, which can trigger misalignment issues, leading to execution failures with severe real-world consequences. To study this issue, we analyze the misalignment-induced vulnerabilities within LLM/VLM-controlled robotic systems and present a mathematical formulation for failure modes arising from variations in input modalities. We propose empirical perturbation strategies to expose these vulnerabilities and validate their effectiveness through experiments on multiple robot manipulation tasks. Our results show that simple input perturbations reduce task execution success rates by 22.2% and 14.6% in two representative LLM/VLM-controlled robotic systems. These findings underscore the importance of input modality robustness and motivate further research to ensure the safe and reliable deployment of advanced LLM/VLM-controlled robotic systems. |
| title | On the Vulnerability of LLM/VLM-Controlled Robotics |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2402.10340 |