Cover Image

Sim2Real-AD: A Modular Sim-to-Real Framework for Deploying VLM-Guided Reinforcement Learning in Real-World Autonomous Driving

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Huang, Zilin, Wan, Zhengyang, Sheng, Zihao, Wang, Boyue, You, Junwei, Leng, Yue, Chen, Sikai
Format:	Preprint
Published:	2026
Subjects:	Robotics Artificial Intelligence Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

DriveVLM-RL: Neuroscience-Inspired Reinforcement Learning with Vision-Language Models for Safe and Deployable Autonomous Driving
by: Huang, Zilin, et al.
Published: (2026)

VLM-RL: A Unified Vision Language Models and Reinforcement Learning Framework for Safe Autonomous Driving
by: Huang, Zilin, et al.
Published: (2024)

VLM-SAFE: Vision-Language Model-Guided Safety-Aware Reinforcement Learning with World Models for Autonomous Driving
by: Qu, Yansong, et al.
Published: (2025)

CurricuVLM: Towards Safe Autonomous Driving via Personalized Safety-Critical Curriculum Learning with Vision-Language Models
by: Sheng, Zihao, et al.
Published: (2025)

Trustworthy Human-AI Collaboration: Reinforcement Learning with Human Feedback and Physics Knowledge for Safe Autonomous Driving
by: Huang, Zilin, et al.
Published: (2024)

Swim2Real: VLM-Guided System Identification for Sim-to-Real Transfer
by: Qiu, Kevin, et al.
Published: (2026)

HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving
by: Huang, Zilin, et al.
Published: (2024)

Dynamics-Decoupled Trajectory Alignment for Sim-to-Real Transfer in Reinforcement Learning for Autonomous Driving
by: Steinecker, Thomas, et al.
Published: (2025)

S2R-Bench: A Sim-to-Real Evaluation Benchmark for Autonomous Driving
by: Wang, Li, et al.
Published: (2025)

Autonomy Oriented Digital Twins for Real2Sim2Real Autoware Deployment
by: Samak, Chinmay Vilas, et al.
Published: (2024)

DexSim2Real: Foundation Model-Guided Sim-to-Real Transfer for Generalizable Dexterous Manipulation
by: Zeng, Zijian, et al.
Published: (2026)

Vision-based DRL Autonomous Driving Agent with Sim2Real Transfer
by: Li, Dianzhao, et al.
Published: (2023)

HyperSim: A Holistic Sim-To-Real Framework For Robust Robotic Manipulation
by: Dong, Junyi, et al.
Published: (2026)

Real-is-Sim: Bridging the Sim-to-Real Gap with a Dynamic Digital Twin
by: Abou-Chakra, Jad, et al.
Published: (2025)

SimScale: Learning to Drive via Real-World Simulation at Scale
by: Tian, Haochen, et al.
Published: (2025)

X-Sim: Cross-Embodiment Learning via Real-to-Sim-to-Real
by: Dan, Prithwish, et al.
Published: (2025)

A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards
by: Patel, Shivansh, et al.
Published: (2025)

Real-to-Sim for Highly Cluttered Environments via Physics-Consistent Inter-Object Reasoning
by: Xiang, Tianyi, et al.
Published: (2026)

Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL
by: Wagenmaker, Andrew, et al.
Published: (2024)

SimShear: Sim-to-Real Shear-based Tactile Servoing
by: Freud, Kipp McAdam, et al.
Published: (2025)

CycleRL: Sim-to-Real Deep Reinforcement Learning for Robust Autonomous Bicycle Control
by: Liu, Gelu, et al.
Published: (2026)

A Platform-Agnostic Deep Reinforcement Learning Framework for Effective Sim2Real Transfer towards Autonomous Driving
by: Li, Dianzhao, et al.
Published: (2023)

Sky-Drive: A Distributed Multi-Agent Simulation Platform for Human-AI Collaborative and Socially-Aware Future Transportation
by: Huang, Zilin, et al.
Published: (2025)

RoboTron-Sim: Improving Real-World Driving via Simulated Hard-Case
by: Xiao, Baihui, et al.
Published: (2025)

Real-to-Sim Grasp: Rethinking the Gap between Simulation and Real World in Grasp Detection
by: Cai, Jia-Feng, et al.
Published: (2024)

Efficient and Reliable Teleoperation through Real-to-Sim-to-Real Shared Autonomy
by: Sha, Shuo, et al.
Published: (2026)

Sim-to-Real Gentle Manipulation of Deformable and Fragile Objects with Stress-Guided Reinforcement Learning
by: Ikemura, Kei, et al.
Published: (2025)

Sim2Real Reinforcement Learning for Soccer skills
by: Spraggett, Jonathan
Published: (2025)

Sim-to-Real Transfer for Mobile Robots with Reinforcement Learning: from NVIDIA Isaac Sim to Gazebo and Real ROS 2 Robots
by: Salimpour, Sahar, et al.
Published: (2025)

Phys2Real: Fusing VLM Priors with Interactive Online Adaptation for Uncertainty-Aware Sim-to-Real Manipulation
by: Wang, Maggie, et al.
Published: (2025)

Sim-to-Real Fruit Detection Using Synthetic Data: Quantitative Evaluation and Embedded Deployment with Isaac Sim
by: Hutter-Mironovova, Martina
Published: (2026)

Sim-to-Real Transfer in Deep Reinforcement Learning for Bipedal Locomotion
by: Bao, Lingfan, et al.
Published: (2025)

Zero-Shot Sim-to-Real Reinforcement Learning for Fruit Harvesting
by: Williams, Emlyn, et al.
Published: (2025)

Betting for Sim-to-Real Performance Evaluation
by: Mahboob, Zaid, et al.
Published: (2026)

Robotic Sim-to-Real Transfer for Long-Horizon Pick-and-Place Tasks in the Robotic Sim2Real Competition
by: Yang, Ming, et al.
Published: (2025)

ReSim: Reliable World Simulation for Autonomous Driving
by: Yang, Jiazhi, et al.
Published: (2025)

Sim2Real Manipulation on Unknown Objects with Tactile-based Reinforcement Learning
by: Su, Entong, et al.
Published: (2024)

VR-Robo: A Real-to-Sim-to-Real Framework for Visual Robot Navigation and Locomotion
by: Zhu, Shaoting, et al.
Published: (2025)

A Schwarz-Christoffel Mapping-based Framework for Sim-to-Real Transfer in Autonomous Robot Operations
by: Gao, Shijie, et al.
Published: (2025)

Sim-to-Real gap in RL: Use Case with TIAGo and Isaac Sim/Gym
by: Albardaner, Jaume, et al.
Published: (2024)