Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918025033154560 |
|---|---|
| author | NVIDIA : Azzolini, Alisson Bai, Junjie Brandon, Hannah Cao, Jiaxin Chattopadhyay, Prithvijit Chen, Huayu Chu, Jinju Cui, Yin Diamond, Jenna Ding, Yifan Feng, Liang Ferroni, Francesco Govindaraju, Rama Gu, Jinwei Gururani, Siddharth Hanafi, Imad El Hao, Zekun Huffman, Jacob Jin, Jingyi Johnson, Brendan Khan, Rizwan Kurian, George Lantz, Elena Lee, Nayeon Li, Zhaoshuo Li, Xuan Liao, Maosheng Lin, Tsung-Yi Lin, Yen-Chen Liu, Ming-Yu Lu, Xiangyu Luo, Alice Mathau, Andrew Ni, Yun Pavao, Lindsey Ping, Wei Romero, David W. Smelyanskiy, Misha Song, Shuran Tchapmi, Lyne Wang, Andrew Z. Wang, Boxin Wang, Haoxiang Wei, Fangyin Xu, Jiashu Xu, Yao Yang, Dinghao Yang, Xiaodong Yang, Zhuolin Zhang, Jingxu Zeng, Xiaohui Zhang, Zhe |
| author_facet | NVIDIA : Azzolini, Alisson Bai, Junjie Brandon, Hannah Cao, Jiaxin Chattopadhyay, Prithvijit Chen, Huayu Chu, Jinju Cui, Yin Diamond, Jenna Ding, Yifan Feng, Liang Ferroni, Francesco Govindaraju, Rama Gu, Jinwei Gururani, Siddharth Hanafi, Imad El Hao, Zekun Huffman, Jacob Jin, Jingyi Johnson, Brendan Khan, Rizwan Kurian, George Lantz, Elena Lee, Nayeon Li, Zhaoshuo Li, Xuan Liao, Maosheng Lin, Tsung-Yi Lin, Yen-Chen Liu, Ming-Yu Lu, Xiangyu Luo, Alice Mathau, Andrew Ni, Yun Pavao, Lindsey Ping, Wei Romero, David W. Smelyanskiy, Misha Song, Shuran Tchapmi, Lyne Wang, Andrew Z. Wang, Boxin Wang, Haoxiang Wei, Fangyin Xu, Jiashu Xu, Yao Yang, Dinghao Yang, Xiaodong Yang, Zhuolin Zhang, Jingxu Zeng, Xiaohui Zhang, Zhe |
| contents | Physical AI systems need to perceive, understand, and perform complex actions in the physical world. In this paper, we present the Cosmos-Reason1 models that can understand the physical world and generate appropriate embodied decisions (e.g., next step action) in natural language through long chain-of-thought reasoning processes. We begin by defining key capabilities for Physical AI reasoning, with a focus on physical common sense and embodied reasoning. To represent physical common sense, we use a hierarchical ontology that captures fundamental knowledge about space, time, and physics. For embodied reasoning, we rely on a two-dimensional ontology that generalizes across different physical embodiments. Building on these capabilities, we develop two multimodal large language models, Cosmos-Reason1-7B and Cosmos-Reason1-56B. We curate data and train our models in two stages: Physical AI supervised fine-tuning (SFT) and Physical AI reinforcement learning (RL). To evaluate our models, we build comprehensive benchmarks for physical common sense and embodied reasoning according to our ontologies. Evaluation results show that Physical AI SFT and RL bring significant improvements. To facilitate the development of Physical AI, we make our code and pre-trained models available under the NVIDIA Open Model License at https://github.com/nvidia-cosmos/cosmos-reason1. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_15558 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning NVIDIA : Azzolini, Alisson Bai, Junjie Brandon, Hannah Cao, Jiaxin Chattopadhyay, Prithvijit Chen, Huayu Chu, Jinju Cui, Yin Diamond, Jenna Ding, Yifan Feng, Liang Ferroni, Francesco Govindaraju, Rama Gu, Jinwei Gururani, Siddharth Hanafi, Imad El Hao, Zekun Huffman, Jacob Jin, Jingyi Johnson, Brendan Khan, Rizwan Kurian, George Lantz, Elena Lee, Nayeon Li, Zhaoshuo Li, Xuan Liao, Maosheng Lin, Tsung-Yi Lin, Yen-Chen Liu, Ming-Yu Lu, Xiangyu Luo, Alice Mathau, Andrew Ni, Yun Pavao, Lindsey Ping, Wei Romero, David W. Smelyanskiy, Misha Song, Shuran Tchapmi, Lyne Wang, Andrew Z. Wang, Boxin Wang, Haoxiang Wei, Fangyin Xu, Jiashu Xu, Yao Yang, Dinghao Yang, Xiaodong Yang, Zhuolin Zhang, Jingxu Zeng, Xiaohui Zhang, Zhe Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning Robotics Physical AI systems need to perceive, understand, and perform complex actions in the physical world. In this paper, we present the Cosmos-Reason1 models that can understand the physical world and generate appropriate embodied decisions (e.g., next step action) in natural language through long chain-of-thought reasoning processes. We begin by defining key capabilities for Physical AI reasoning, with a focus on physical common sense and embodied reasoning. To represent physical common sense, we use a hierarchical ontology that captures fundamental knowledge about space, time, and physics. For embodied reasoning, we rely on a two-dimensional ontology that generalizes across different physical embodiments. Building on these capabilities, we develop two multimodal large language models, Cosmos-Reason1-7B and Cosmos-Reason1-56B. We curate data and train our models in two stages: Physical AI supervised fine-tuning (SFT) and Physical AI reinforcement learning (RL). To evaluate our models, we build comprehensive benchmarks for physical common sense and embodied reasoning according to our ontologies. Evaluation results show that Physical AI SFT and RL bring significant improvements. To facilitate the development of Physical AI, we make our code and pre-trained models available under the NVIDIA Open Model License at https://github.com/nvidia-cosmos/cosmos-reason1. |
| title | Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning |
| topic | Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning Robotics |
| url | https://arxiv.org/abs/2503.15558 |