Adversarial Attacks on Robotic Vision Language Action Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915323743043584 |
|---|---|
| author | Jones, Eliot Krzysztof Robey, Alexander Zou, Andy Ravichandran, Zachary Pappas, George J. Hassani, Hamed Fredrikson, Matt Kolter, J. Zico |
| author_facet | Jones, Eliot Krzysztof Robey, Alexander Zou, Andy Ravichandran, Zachary Pappas, George J. Hassani, Hamed Fredrikson, Matt Kolter, J. Zico |
| contents | The emergence of vision-language-action models (VLAs) for end-to-end control is reshaping the field of robotics by enabling the fusion of multimodal sensory inputs at the billion-parameter scale. The capabilities of VLAs stem primarily from their architectures, which are often based on frontier large language models (LLMs). However, LLMs are known to be susceptible to adversarial misuse, and given the significant physical risks inherent to robotics, questions remain regarding the extent to which VLAs inherit these vulnerabilities. Motivated by these concerns, in this work we initiate the study of adversarial attacks on VLA-controlled robots. Our main algorithmic contribution is the adaptation and application of LLM jailbreaking attacks to obtain complete control authority over VLAs. We find that textual attacks, which are applied once at the beginning of a rollout, facilitate full reachability of the action space of commonly used VLAs and often persist over longer horizons. This differs significantly from LLM jailbreaking literature, as attacks in the real world do not have to be semantically linked to notions of harm. We make all code available at https://github.com/eliotjones1/robogcg . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_03350 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Adversarial Attacks on Robotic Vision Language Action Models Jones, Eliot Krzysztof Robey, Alexander Zou, Andy Ravichandran, Zachary Pappas, George J. Hassani, Hamed Fredrikson, Matt Kolter, J. Zico Robotics Artificial Intelligence The emergence of vision-language-action models (VLAs) for end-to-end control is reshaping the field of robotics by enabling the fusion of multimodal sensory inputs at the billion-parameter scale. The capabilities of VLAs stem primarily from their architectures, which are often based on frontier large language models (LLMs). However, LLMs are known to be susceptible to adversarial misuse, and given the significant physical risks inherent to robotics, questions remain regarding the extent to which VLAs inherit these vulnerabilities. Motivated by these concerns, in this work we initiate the study of adversarial attacks on VLA-controlled robots. Our main algorithmic contribution is the adaptation and application of LLM jailbreaking attacks to obtain complete control authority over VLAs. We find that textual attacks, which are applied once at the beginning of a rollout, facilitate full reachability of the action space of commonly used VLAs and often persist over longer horizons. This differs significantly from LLM jailbreaking literature, as attacks in the real world do not have to be semantically linked to notions of harm. We make all code available at https://github.com/eliotjones1/robogcg . |
| title | Adversarial Attacks on Robotic Vision Language Action Models |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2506.03350 |