Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.26848 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917451971690496 |
|---|---|
| author | Tian, Yuxuan Jin, Yurun Yu, Bin Shi, Yukun Wu, Hao Liu, Chi Harold Chen, Kai Huang, Cong |
| author_facet | Tian, Yuxuan Jin, Yurun Yu, Bin Shi, Yukun Wu, Hao Liu, Chi Harold Chen, Kai Huang, Cong |
| contents | Robotic manipulation requires reasoning about future spatial-temporal interactions and geometric constraints, yet existing Vision-Language-Action (VLA) policies often leave predictive representation weakly coupled with action execution, causing failures in tasks requiring precise spatial-temporal coordination. We propose STARRY, a world-model-enhanced action-generation policy that aligns spatial-temporal prediction and action generation by jointly denoising future spatial-temporal latents and actions through a unified diffusion process. To bridge 2D visual tokens and 3D metric control, STARRY introduces Geometry-Aware Selective Attention Modulation (GASAM), which converts predicted depth and end-effector geometry into token-aligned weights for selective action-attention modulation. On RoboTwin 2.0, STARRY achieves 93.82% / 93.30% average success under Clean and Randomized settings across 50 bimanual tasks. Real-world experiments show that STARRY improves average success from 42.5% to 70.8% compared with $π_{0.5}$. These results demonstrate the effectiveness of action-centric spatial-temporal world modeling for spatially and temporally demanding robotic manipulation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_26848 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | STARRY: Spatial-Temporal Action-Centric World Modeling for Robotic Manipulation Tian, Yuxuan Jin, Yurun Yu, Bin Shi, Yukun Wu, Hao Liu, Chi Harold Chen, Kai Huang, Cong Robotics Robotic manipulation requires reasoning about future spatial-temporal interactions and geometric constraints, yet existing Vision-Language-Action (VLA) policies often leave predictive representation weakly coupled with action execution, causing failures in tasks requiring precise spatial-temporal coordination. We propose STARRY, a world-model-enhanced action-generation policy that aligns spatial-temporal prediction and action generation by jointly denoising future spatial-temporal latents and actions through a unified diffusion process. To bridge 2D visual tokens and 3D metric control, STARRY introduces Geometry-Aware Selective Attention Modulation (GASAM), which converts predicted depth and end-effector geometry into token-aligned weights for selective action-attention modulation. On RoboTwin 2.0, STARRY achieves 93.82% / 93.30% average success under Clean and Randomized settings across 50 bimanual tasks. Real-world experiments show that STARRY improves average success from 42.5% to 70.8% compared with $π_{0.5}$. These results demonstrate the effectiveness of action-centric spatial-temporal world modeling for spatially and temporally demanding robotic manipulation. |
| title | STARRY: Spatial-Temporal Action-Centric World Modeling for Robotic Manipulation |
| topic | Robotics |
| url | https://arxiv.org/abs/2604.26848 |