Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.06157 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911045623218176 |
|---|---|
| author | Li, William Hamilton, Lei Al-natour, Kaise Mohindra, Sanjeev |
| author_facet | Li, William Hamilton, Lei Al-natour, Kaise Mohindra, Sanjeev |
| contents | This paper focuses on evaluating the effectiveness of Large Language Models at solving embodied robotic tasks using the Meta PARTNER benchmark. Meta PARTNR provides simplified environments and robotic interactions within randomized indoor kitchen scenes. Each randomized kitchen scene is given a task where two robotic agents cooperatively work together to solve the task. We evaluated multiple frontier models on Meta PARTNER environments. Our results indicate that reasoning models like OpenAI o3-mini outperform non-reasoning models like OpenAI GPT-4o and Llama 3 when operating in PARTNR's robotic embodied environments. o3-mini displayed outperform across centralized, decentralized, full observability, and partial observability configurations. This provides a promising avenue of research for embodied robotic development. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_06157 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Evaluation of Habitat Robotics using Large Language Models Li, William Hamilton, Lei Al-natour, Kaise Mohindra, Sanjeev Robotics Computation and Language This paper focuses on evaluating the effectiveness of Large Language Models at solving embodied robotic tasks using the Meta PARTNER benchmark. Meta PARTNR provides simplified environments and robotic interactions within randomized indoor kitchen scenes. Each randomized kitchen scene is given a task where two robotic agents cooperatively work together to solve the task. We evaluated multiple frontier models on Meta PARTNER environments. Our results indicate that reasoning models like OpenAI o3-mini outperform non-reasoning models like OpenAI GPT-4o and Llama 3 when operating in PARTNR's robotic embodied environments. o3-mini displayed outperform across centralized, decentralized, full observability, and partial observability configurations. This provides a promising avenue of research for embodied robotic development. |
| title | Evaluation of Habitat Robotics using Large Language Models |
| topic | Robotics Computation and Language |
| url | https://arxiv.org/abs/2507.06157 |