Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.09971 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910048183123968 |
|---|---|
| author | Shen, William Kumar, Nishanth Chintalapudi, Sahit Wang, Jie Watson, Christopher Hu, Edward Cao, Jing Jayaraman, Dinesh Kaelbling, Leslie Pack Lozano-Pérez, Tomás |
| author_facet | Shen, William Kumar, Nishanth Chintalapudi, Sahit Wang, Jie Watson, Christopher Hu, Edward Cao, Jing Jayaraman, Dinesh Kaelbling, Leslie Pack Lozano-Pérez, Tomás |
| contents | We present TiPToP, an extensible modular system that combines pretrained vision foundation models with an existing Task and Motion Planner (TAMP) to solve multi-step manipulation tasks directly from input RGB images and natural-language instructions. Our system aims to be simple and easy-to-use: it can be installed and run on a standard DROID setup in under one hour and adapted to new embodiments with minimal effort. We evaluate TiPToP -- which requires zero robot data -- over 28 tabletop manipulation tasks in simulation and the real world and find it matches or outperforms $π_{0.5}\text{-DROID}$, a vision-language-action (VLA) model fine-tuned on 350 hours of embodiment-specific demonstrations. TiPToP's modular architecture enables us to analyze the system's failure modes at the component level. We analyze results from an evaluation of 173 trials and identify directions for improvement. We release TiPToP open-source to further research on modular manipulation systems and tighter integration between learning and planning. Project website and code: https://tiptop-robot.github.io |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_09971 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | TiPToP: A Modular Open-Vocabulary Planning System for Robotic Manipulation Shen, William Kumar, Nishanth Chintalapudi, Sahit Wang, Jie Watson, Christopher Hu, Edward Cao, Jing Jayaraman, Dinesh Kaelbling, Leslie Pack Lozano-Pérez, Tomás Robotics We present TiPToP, an extensible modular system that combines pretrained vision foundation models with an existing Task and Motion Planner (TAMP) to solve multi-step manipulation tasks directly from input RGB images and natural-language instructions. Our system aims to be simple and easy-to-use: it can be installed and run on a standard DROID setup in under one hour and adapted to new embodiments with minimal effort. We evaluate TiPToP -- which requires zero robot data -- over 28 tabletop manipulation tasks in simulation and the real world and find it matches or outperforms $π_{0.5}\text{-DROID}$, a vision-language-action (VLA) model fine-tuned on 350 hours of embodiment-specific demonstrations. TiPToP's modular architecture enables us to analyze the system's failure modes at the component level. We analyze results from an evaluation of 173 trials and identify directions for improvement. We release TiPToP open-source to further research on modular manipulation systems and tighter integration between learning and planning. Project website and code: https://tiptop-robot.github.io |
| title | TiPToP: A Modular Open-Vocabulary Planning System for Robotic Manipulation |
| topic | Robotics |
| url | https://arxiv.org/abs/2603.09971 |