Any3D-VLA: Enhancing VLA Robustness via Diverse Point Clouds
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917495157293056 |
|---|---|
| author | Fan, Xianzhe Deng, Shengliang Wu, Xiaoyang Lu, Yuxiang Li, Zhuoling Yan, Mi Zhang, Yujia Zhang, Zhizheng Wang, He Zhao, Hengshuang |
| author_facet | Fan, Xianzhe Deng, Shengliang Wu, Xiaoyang Lu, Yuxiang Li, Zhuoling Yan, Mi Zhang, Yujia Zhang, Zhizheng Wang, He Zhao, Hengshuang |
| contents | Existing Vision-Language-Action (VLA) models typically take 2D images as visual input, which limits their spatial understanding in complex scenes. How can we incorporate 3D information to enhance VLA capabilities? We conduct a pilot study across different observation spaces and visual representations. The results show that explicitly lifting visual input into point clouds yields representations that better complement their corresponding 2D representations. To address the challenges of (1) scarce 3D data and (2) the domain gap induced by cross-environment differences and depth-scale biases, we propose Any3D-VLA. It unifies the simulator, sensor, and model-estimated point clouds within a training pipeline, constructs diverse inputs, and learns domain-agnostic 3D representations that are fused with the corresponding 2D representations. Simulation and real-world experiments demonstrate Any3D-VLA's advantages in improving performance and mitigating the domain gap. Our project homepage is available at https://xianzhefan.github.io/Any3D-VLA.github.io. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_00807 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Any3D-VLA: Enhancing VLA Robustness via Diverse Point Clouds Fan, Xianzhe Deng, Shengliang Wu, Xiaoyang Lu, Yuxiang Li, Zhuoling Yan, Mi Zhang, Yujia Zhang, Zhizheng Wang, He Zhao, Hengshuang Computer Vision and Pattern Recognition Robotics Existing Vision-Language-Action (VLA) models typically take 2D images as visual input, which limits their spatial understanding in complex scenes. How can we incorporate 3D information to enhance VLA capabilities? We conduct a pilot study across different observation spaces and visual representations. The results show that explicitly lifting visual input into point clouds yields representations that better complement their corresponding 2D representations. To address the challenges of (1) scarce 3D data and (2) the domain gap induced by cross-environment differences and depth-scale biases, we propose Any3D-VLA. It unifies the simulator, sensor, and model-estimated point clouds within a training pipeline, constructs diverse inputs, and learns domain-agnostic 3D representations that are fused with the corresponding 2D representations. Simulation and real-world experiments demonstrate Any3D-VLA's advantages in improving performance and mitigating the domain gap. Our project homepage is available at https://xianzhefan.github.io/Any3D-VLA.github.io. |
| title | Any3D-VLA: Enhancing VLA Robustness via Diverse Point Clouds |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2602.00807 |