Red-Teaming Vision-Language-Action Models via Quality Diversity Prompt Generation for Robust Robot Policies
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914444459638784 |
|---|---|
| author | Srikanth, Siddharth Liang, Freddie Hsu, Ya-Chuan Bhatt, Varun Zhao, Shihan Chen, Henry Tjanaka, Bryon Hwang, Minjune Saran, Akanksha Seita, Daniel Tabrez, Aaquib Nikolaidis, Stefanos |
| author_facet | Srikanth, Siddharth Liang, Freddie Hsu, Ya-Chuan Bhatt, Varun Zhao, Shihan Chen, Henry Tjanaka, Bryon Hwang, Minjune Saran, Akanksha Seita, Daniel Tabrez, Aaquib Nikolaidis, Stefanos |
| contents | Vision-Language-Action (VLA) models have significant potential to enable general-purpose robotic systems for a range of vision-language tasks. However, the performance of VLA-based robots is highly sensitive to the precise wording of language instructions, and it remains difficult to predict when such robots will fail. We propose Quality Diversity (QD) optimization as a natural framework for red-teaming embodied models, and present Q-DIG (Quality Diversity for Diverse Instruction Generation), which performs red-teaming by scalably identifying diverse, natural language task descriptions that induce failures while remaining task-relevant. Q-DIG integrates QD techniques with Vision-Language Models (VLMs) to generate a broad spectrum of adversarial instructions that expose meaningful vulnerabilities in VLA behavior. Our results across multiple simulation benchmarks show that Q-DIG finds more diverse and meaningful failure modes compared to baseline methods, and that fine-tuning VLAs on the generated instructions improves task success rates. Furthermore, results from a user study highlight that Q-DIG generates prompts judged to be more natural and human-like than those from baselines. Finally, real-world evaluations of Q-DIG prompts show results consistent with simulation, and fine-tuning VLAs on the generated prompts further success rates on unseen instructions. Together, these findings suggest that Q-DIG is a promising approach for identifying vulnerabilities and improving the robustness of VLA-based robots. Our anonymous project website is at qdigvla.github.io. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_12510 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Red-Teaming Vision-Language-Action Models via Quality Diversity Prompt Generation for Robust Robot Policies Srikanth, Siddharth Liang, Freddie Hsu, Ya-Chuan Bhatt, Varun Zhao, Shihan Chen, Henry Tjanaka, Bryon Hwang, Minjune Saran, Akanksha Seita, Daniel Tabrez, Aaquib Nikolaidis, Stefanos Robotics Artificial Intelligence Computation and Language Vision-Language-Action (VLA) models have significant potential to enable general-purpose robotic systems for a range of vision-language tasks. However, the performance of VLA-based robots is highly sensitive to the precise wording of language instructions, and it remains difficult to predict when such robots will fail. We propose Quality Diversity (QD) optimization as a natural framework for red-teaming embodied models, and present Q-DIG (Quality Diversity for Diverse Instruction Generation), which performs red-teaming by scalably identifying diverse, natural language task descriptions that induce failures while remaining task-relevant. Q-DIG integrates QD techniques with Vision-Language Models (VLMs) to generate a broad spectrum of adversarial instructions that expose meaningful vulnerabilities in VLA behavior. Our results across multiple simulation benchmarks show that Q-DIG finds more diverse and meaningful failure modes compared to baseline methods, and that fine-tuning VLAs on the generated instructions improves task success rates. Furthermore, results from a user study highlight that Q-DIG generates prompts judged to be more natural and human-like than those from baselines. Finally, real-world evaluations of Q-DIG prompts show results consistent with simulation, and fine-tuning VLAs on the generated prompts further success rates on unseen instructions. Together, these findings suggest that Q-DIG is a promising approach for identifying vulnerabilities and improving the robustness of VLA-based robots. Our anonymous project website is at qdigvla.github.io. |
| title | Red-Teaming Vision-Language-Action Models via Quality Diversity Prompt Generation for Robust Robot Policies |
| topic | Robotics Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2603.12510 |