SimWorld: An Open-ended Realistic Simulator for Autonomous Agents in Physical and Social Worlds

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ren, Jiawei, Zhuang, Yan, Ye, Xiaokang, Mao, Lingjun, He, Xuhong, Shen, Jianzhi, Dogra, Mrinaal, Liang, Yiming, Zhang, Ruixuan, Yue, Tianai, Yang, Yiqing, Liu, Eric, Wu, Ryan, Benavente, Kevin, Nagaraju, Rajiv Mandya, Faayez, Muhammad, Zhang, Xiyan, Sharma, Dhruv Vivek, Zhong, Xianrui, Ma, Ziqiao, Shu, Tianmin, Hu, Zhiting, Qin, Lianhui
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866915749140889600
author	Ren, Jiawei Zhuang, Yan Ye, Xiaokang Mao, Lingjun He, Xuhong Shen, Jianzhi Dogra, Mrinaal Liang, Yiming Zhang, Ruixuan Yue, Tianai Yang, Yiqing Liu, Eric Wu, Ryan Benavente, Kevin Nagaraju, Rajiv Mandya Faayez, Muhammad Zhang, Xiyan Sharma, Dhruv Vivek Zhong, Xianrui Ma, Ziqiao Shu, Tianmin Hu, Zhiting Qin, Lianhui
author_facet	Ren, Jiawei Zhuang, Yan Ye, Xiaokang Mao, Lingjun He, Xuhong Shen, Jianzhi Dogra, Mrinaal Liang, Yiming Zhang, Ruixuan Yue, Tianai Yang, Yiqing Liu, Eric Wu, Ryan Benavente, Kevin Nagaraju, Rajiv Mandya Faayez, Muhammad Zhang, Xiyan Sharma, Dhruv Vivek Zhong, Xianrui Ma, Ziqiao Shu, Tianmin Hu, Zhiting Qin, Lianhui
contents	While LLM/VLM-powered AI agents have advanced rapidly in math, coding, and computer use, their applications in complex physical and social environments remain challenging. Building agents that can survive and thrive in the real world (for example, by autonomously earning income or running a business) requires massive-scale interaction, reasoning, training, and evaluation across diverse embodied scenarios. However, existing world simulators for such development fall short: they often rely on limited hand-crafted environments, simulate simplified game-like physics and social rules, and lack native support for LLM/VLM agents. We introduce SimWorld, a new simulator built on Unreal Engine 5, designed for developing and evaluating LLM/VLM agents in rich, real-world-like settings. SimWorld offers three core capabilities: (1) realistic, open-ended world simulation, including accurate physical and social dynamics and language-driven procedural environment generation; (2) a rich interface for LLM/VLM agents, with multimodal world inputs and open-vocabulary actions at varying levels of abstraction; and (3) diverse and extensible physical and social reasoning scenarios that are easily customizable by users. We demonstrate SimWorld by deploying frontier LLM agents (e.g., GPT-4o, Gemini-2.5-Flash, Claude-3.5, and DeepSeek-Prover-V2) on long-horizon multi-agent delivery tasks involving strategic cooperation and competition. The results reveal distinct reasoning patterns and limitations across models. We open-source SimWorld and hope it becomes a foundational platform for advancing real-world agent intelligence across disciplines: https://simworld.org.
format	Preprint
id	arxiv_https___arxiv_org_abs_2512_01078
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	SimWorld: An Open-ended Realistic Simulator for Autonomous Agents in Physical and Social Worlds Ren, Jiawei Zhuang, Yan Ye, Xiaokang Mao, Lingjun He, Xuhong Shen, Jianzhi Dogra, Mrinaal Liang, Yiming Zhang, Ruixuan Yue, Tianai Yang, Yiqing Liu, Eric Wu, Ryan Benavente, Kevin Nagaraju, Rajiv Mandya Faayez, Muhammad Zhang, Xiyan Sharma, Dhruv Vivek Zhong, Xianrui Ma, Ziqiao Shu, Tianmin Hu, Zhiting Qin, Lianhui Artificial Intelligence While LLM/VLM-powered AI agents have advanced rapidly in math, coding, and computer use, their applications in complex physical and social environments remain challenging. Building agents that can survive and thrive in the real world (for example, by autonomously earning income or running a business) requires massive-scale interaction, reasoning, training, and evaluation across diverse embodied scenarios. However, existing world simulators for such development fall short: they often rely on limited hand-crafted environments, simulate simplified game-like physics and social rules, and lack native support for LLM/VLM agents. We introduce SimWorld, a new simulator built on Unreal Engine 5, designed for developing and evaluating LLM/VLM agents in rich, real-world-like settings. SimWorld offers three core capabilities: (1) realistic, open-ended world simulation, including accurate physical and social dynamics and language-driven procedural environment generation; (2) a rich interface for LLM/VLM agents, with multimodal world inputs and open-vocabulary actions at varying levels of abstraction; and (3) diverse and extensible physical and social reasoning scenarios that are easily customizable by users. We demonstrate SimWorld by deploying frontier LLM agents (e.g., GPT-4o, Gemini-2.5-Flash, Claude-3.5, and DeepSeek-Prover-V2) on long-horizon multi-agent delivery tasks involving strategic cooperation and competition. The results reveal distinct reasoning patterns and limitations across models. We open-source SimWorld and hope it becomes a foundational platform for advancing real-world agent intelligence across disciplines: https://simworld.org.
title	SimWorld: An Open-ended Realistic Simulator for Autonomous Agents in Physical and Social Worlds
topic	Artificial Intelligence
url	https://arxiv.org/abs/2512.01078