RoboTrustBench: Benchmarking the Trustworthiness of Video World Models for Robotic Manipulation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913177972768768 |
|---|---|
| author | Li, Huiqiong Wang, Jiayu Mei, Zhiting Majumdar, Anirudha Chen, Jingjing Zhu, Bin |
| author_facet | Li, Huiqiong Wang, Jiayu Mei, Zhiting Majumdar, Anirudha Chen, Jingjing Zhu, Bin |
| contents | Video world models are increasingly used in robotic manipulation, yet existing benchmarks mostly evaluate them under valid, feasible, and safe instructions. We introduce RoboTrustBench, a benchmark for evaluating the trustworthiness of video world models under four scenarios: Normal, Constraint-Sensitive, Counterfactual, and Adversarial. Built from real-world DROID episodes, RoboTrustBench contains 1,207 expert-validated instruction-image pairs and a six-dimensional evaluation protocol with 13 fine-grained criteria. Evaluating seven representative video world models with human and MLLM assessment, we find that current models often generate visually coherent videos, but struggle with constraint reasoning, counterfactual grounding, physical interaction, and unsafe-instruction suppression. These results show that visual quality and surface-level instruction following are insufficient for trustworthy robotic video world modeling. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2606_01600 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | RoboTrustBench: Benchmarking the Trustworthiness of Video World Models for Robotic Manipulation Li, Huiqiong Wang, Jiayu Mei, Zhiting Majumdar, Anirudha Chen, Jingjing Zhu, Bin Computer Vision and Pattern Recognition Computation and Language Robotics Video world models are increasingly used in robotic manipulation, yet existing benchmarks mostly evaluate them under valid, feasible, and safe instructions. We introduce RoboTrustBench, a benchmark for evaluating the trustworthiness of video world models under four scenarios: Normal, Constraint-Sensitive, Counterfactual, and Adversarial. Built from real-world DROID episodes, RoboTrustBench contains 1,207 expert-validated instruction-image pairs and a six-dimensional evaluation protocol with 13 fine-grained criteria. Evaluating seven representative video world models with human and MLLM assessment, we find that current models often generate visually coherent videos, but struggle with constraint reasoning, counterfactual grounding, physical interaction, and unsafe-instruction suppression. These results show that visual quality and surface-level instruction following are insufficient for trustworthy robotic video world modeling. |
| title | RoboTrustBench: Benchmarking the Trustworthiness of Video World Models for Robotic Manipulation |
| topic | Computer Vision and Pattern Recognition Computation and Language Robotics |
| url | https://arxiv.org/abs/2606.01600 |