Cover Image

A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Patel, Shivansh, Yin, Xinchen, Huang, Wenlong, Garg, Shubham, Nayyeri, Hooshang, Fei-Fei, Li, Lazebnik, Svetlana, Li, Yunzhu
Format:	Preprint
Published:	2025
Subjects:	Robotics Artificial Intelligence Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrations
by: Patel, Shivansh, et al.
Published: (2025)

ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation
by: Huang, Wenlong, et al.
Published: (2024)

RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulation
by: Jiang, Hanxiao, et al.
Published: (2024)

KUDA: Keypoints to Unify Dynamics Learning and Visual Prompting for Open-Vocabulary Robotic Manipulation
by: Liu, Zixian, et al.
Published: (2025)

DexSim2Real: Foundation Model-Guided Sim-to-Real Transfer for Generalizable Dexterous Manipulation
by: Zeng, Zijian, et al.
Published: (2026)

Efficient and Reliable Teleoperation through Real-to-Sim-to-Real Shared Autonomy
by: Sha, Shuo, et al.
Published: (2026)

Real-to-Sim Deformable Object Manipulation: Optimizing Physics Models with Residual Mappings for Robotic Surgery
by: Liang, Xiao, et al.
Published: (2023)

CRAFT: A Tendon-Driven Hand with Hybrid Hard-Soft Compliance
by: Lin, Leo, et al.
Published: (2026)

HyperSim: A Holistic Sim-To-Real Framework For Robust Robotic Manipulation
by: Dong, Junyi, et al.
Published: (2026)

Can Real-to-Sim Approaches Capture Dynamic Fabric Behavior for Robotic Fabric Manipulation?
by: Ru, Yingdong, et al.
Published: (2025)

REALM: A Real-to-Sim Validated Benchmark for Generalization in Robotic Manipulation
by: Sedlacek, Martin, et al.
Published: (2025)

Phys2Real: Fusing VLM Priors with Interactive Online Adaptation for Uncertainty-Aware Sim-to-Real Manipulation
by: Wang, Maggie, et al.
Published: (2025)

Interactive World Simulator for Robot Policy Training and Evaluation
by: Wang, Yixuan, et al.
Published: (2026)

Swim2Real: VLM-Guided System Identification for Sim-to-Real Transfer
by: Qiu, Kevin, et al.
Published: (2026)

Reinforced Embodied Planning with Verifiable Reward for Real-World Robotic Manipulation
by: Bo, Zitong, et al.
Published: (2025)

Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation
by: Torne, Marcel, et al.
Published: (2024)

TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction
by: Jiang, Yunfan, et al.
Published: (2024)

UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulation
by: Tang, Yihe, et al.
Published: (2025)

Re$^3$Sim: Generating High-Fidelity Simulation Data via 3D-Photorealistic Real-to-Sim for Robotic Manipulation
by: Han, Xiaoshen, et al.
Published: (2025)

In-Hand Manipulation of Articulated Tools with Dexterous Robot Hands with Sim-to-Real Transfer
by: Atar, Soofiyan, et al.
Published: (2025)

Impact-Aware Robotic Manipulation: Quantifying the Sim-To-Real Gap for Velocity Jumps
by: van Steen, Jari, et al.
Published: (2024)

ProcVLM: Learning Procedure-Grounded Progress Rewards for Robotic Manipulation
by: Feng, Youhe, et al.
Published: (2026)

On the Role of the Action Space in Robot Manipulation Learning and Sim-to-Real Transfer
by: Aljalbout, Elie, et al.
Published: (2023)

Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactions
by: Zhang, Kaifeng, et al.
Published: (2025)

Robotic Sim-to-Real Transfer for Long-Horizon Pick-and-Place Tasks in the Robotic Sim2Real Competition
by: Yang, Ming, et al.
Published: (2025)

Triple Regression for Camera Agnostic Sim2Real Robot Grasping and Manipulation Tasks
by: Zeng, Yuanhong, et al.
Published: (2023)

ExoGS: A 4D Real-to-Sim-to-Real Framework for Scalable Manipulation Data Collection
by: Wang, Yiming, et al.
Published: (2026)

Sim-To-Real Transfer for Visual Reinforcement Learning of Deformable Object Manipulation for Robot-Assisted Surgery
by: Scheikl, Paul Maria, et al.
Published: (2024)

Zero-Shot Sim-to-Real Robot Learning: A Dexterous Manipulation Study on Reactive Catching
by: Ren, Kejia, et al.
Published: (2026)

RL-GSBridge: 3D Gaussian Splatting Based Real2Sim2Real Method for Robotic Manipulation Learning
by: Wu, Yuxuan, et al.
Published: (2024)

Best of Sim and Real: Decoupled Visuomotor Manipulation via Learning Control in Simulation and Perception in Real
by: Huang, Jialei, et al.
Published: (2025)

Dex4D: Task-Agnostic Point Track Policy for Sim-to-Real Dexterous Manipulation
by: Kuang, Yuxuan, et al.
Published: (2026)

VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation
by: He, Tairan, et al.
Published: (2025)

ManipulationNet: An Infrastructure for Benchmarking Real-World Robot Manipulation with Physical Skill Challenges and Embodied Multimodal Reasoning
by: Chen, Yiting, et al.
Published: (2026)

3D Affordance Keypoint Detection for Robotic Manipulation
by: Liu, Zhiyang, et al.
Published: (2025)

TwinAligner: Visual-Dynamic Alignment Empowers Physics-aware Real2Sim2Real for Robotic Manipulation
by: Fan, Hongwei, et al.
Published: (2025)

BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activities
by: Jiang, Yunfan, et al.
Published: (2025)

Rethinking Intermediate Representation for VLM-based Robot Manipulation
by: Tang, Weiliang, et al.
Published: (2025)

Sim-to-Real of Soft Robots with Learned Residual Physics
by: Gao, Junpeng, et al.
Published: (2024)

Sim-to-Real Gentle Manipulation of Deformable and Fragile Objects with Stress-Guided Reinforcement Learning
by: Ikemura, Kei, et al.
Published: (2025)