Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.23055 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915812795744256 |
|---|---|
| author | Zhang, Ruoxuan Zheng, Qiyun Zhou, Zhiyu Liao, Ziqi Wu, Siyu Jiang-Lin, Jian-Yu Wen, Bin Xie, Hongxia Fu, Jianlong Cheng, Wen-Huang |
| author_facet | Zhang, Ruoxuan Zheng, Qiyun Zhou, Zhiyu Liao, Ziqi Wu, Siyu Jiang-Lin, Jian-Yu Wen, Bin Xie, Hongxia Fu, Jianlong Cheng, Wen-Huang |
| contents | Theory of Mind (ToM) refers to the ability to infer others' mental states, such as beliefs, desires, and intentions. Current vision-language embodied agents lack ToM-based decision-making, and existing benchmarks focus solely on human mental states while ignoring the agent's own perspective, hindering coherent decision and action generation. To address this, we propose MindPower, a Robot-Centric framework integrating Perception, Mental Reasoning, Decision Making and Action. Given multimodal inputs, MindPower first perceives the environment and human states, then performs ToM Reasoning to model both self and others, and finally generates decisions and actions guided by inferred mental states. Furthermore, we introduce Mind-Reward, a novel optimization objective that encourages VLMs to produce consistent ToM Reasoning and behavior. Our model outperforms GPT-4o by 12.77% in decision making and 12.49% in action generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_23055 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MindPower: Enabling Theory-of-Mind Reasoning in VLM-based Embodied Agents Zhang, Ruoxuan Zheng, Qiyun Zhou, Zhiyu Liao, Ziqi Wu, Siyu Jiang-Lin, Jian-Yu Wen, Bin Xie, Hongxia Fu, Jianlong Cheng, Wen-Huang Artificial Intelligence Theory of Mind (ToM) refers to the ability to infer others' mental states, such as beliefs, desires, and intentions. Current vision-language embodied agents lack ToM-based decision-making, and existing benchmarks focus solely on human mental states while ignoring the agent's own perspective, hindering coherent decision and action generation. To address this, we propose MindPower, a Robot-Centric framework integrating Perception, Mental Reasoning, Decision Making and Action. Given multimodal inputs, MindPower first perceives the environment and human states, then performs ToM Reasoning to model both self and others, and finally generates decisions and actions guided by inferred mental states. Furthermore, we introduce Mind-Reward, a novel optimization objective that encourages VLMs to produce consistent ToM Reasoning and behavior. Our model outperforms GPT-4o by 12.77% in decision making and 12.49% in action generation. |
| title | MindPower: Enabling Theory-of-Mind Reasoning in VLM-based Embodied Agents |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2511.23055 |