Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.09920 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914339475161088 |
|---|---|
| author | Huang, Ruopeng Yang, Boyu Gui, Wenlong Morgan, Jeremy Biyik, Erdem Li, Jiachen |
| author_facet | Huang, Ruopeng Yang, Boyu Gui, Wenlong Morgan, Jeremy Biyik, Erdem Li, Jiachen |
| contents | Accurate and safe robotic manipulation under dynamic and visually occluded conditions remains a core challenge in real-world deployment. We introduce SyncTwin, a novel digital twin framework that unifies fast 3D scene reconstruction and real-to-sim synchronization for robust and safety-aware robotic manipulation in such environments. In the offline stage, we employ VGGT to rapidly reconstruct object-level 3D assets from RGB images, forming a reusable geometry library. During execution, SyncTwin continuously synchronizes the digital twin by tracking real-world object states via point cloud segmentation updates and aligning them through colored-ICP registration. The synchronized twin enables motion planners to compute collision-free and dynamically feasible trajectories in simulation, which are safely executed on the real robot through a closed real-to-sim-to-real loop. Experiments in dynamic and occluded scenes show that SyncTwin improves manipulation performance and motion safety, demonstrating the effectiveness of digital twin synchronization for real-world robotic execution. The video demos and code can be found on the project website: https://sync-twin.github.io/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_09920 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SyncTwin: Fast Digital Twin Construction and Synchronization for Safe Robotic Manipulation Huang, Ruopeng Yang, Boyu Gui, Wenlong Morgan, Jeremy Biyik, Erdem Li, Jiachen Robotics Accurate and safe robotic manipulation under dynamic and visually occluded conditions remains a core challenge in real-world deployment. We introduce SyncTwin, a novel digital twin framework that unifies fast 3D scene reconstruction and real-to-sim synchronization for robust and safety-aware robotic manipulation in such environments. In the offline stage, we employ VGGT to rapidly reconstruct object-level 3D assets from RGB images, forming a reusable geometry library. During execution, SyncTwin continuously synchronizes the digital twin by tracking real-world object states via point cloud segmentation updates and aligning them through colored-ICP registration. The synchronized twin enables motion planners to compute collision-free and dynamically feasible trajectories in simulation, which are safely executed on the real robot through a closed real-to-sim-to-real loop. Experiments in dynamic and occluded scenes show that SyncTwin improves manipulation performance and motion safety, demonstrating the effectiveness of digital twin synchronization for real-world robotic execution. The video demos and code can be found on the project website: https://sync-twin.github.io/. |
| title | SyncTwin: Fast Digital Twin Construction and Synchronization for Safe Robotic Manipulation |
| topic | Robotics |
| url | https://arxiv.org/abs/2601.09920 |