Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915335918059520 |
|---|---|
| author | Zhang, Lingfeng Liu, Yuecheng Zhang, Zhanguang Aghaei, Matin Hu, Yaochen Gu, Hongjian Alomrani, Mohammad Ali Bravo, David Gamaliel Arcos Karimi, Raika Hamidizadeh, Atia Xu, Haoping Huang, Guowei Zhang, Zhanpeng Cao, Tongtong Qiu, Weichao Quan, Xingyue Hao, Jianye Zhuang, Yuzheng Zhang, Yingxue |
| author_facet | Zhang, Lingfeng Liu, Yuecheng Zhang, Zhanguang Aghaei, Matin Hu, Yaochen Gu, Hongjian Alomrani, Mohammad Ali Bravo, David Gamaliel Arcos Karimi, Raika Hamidizadeh, Atia Xu, Haoping Huang, Guowei Zhang, Zhanpeng Cao, Tongtong Qiu, Weichao Quan, Xingyue Hao, Jianye Zhuang, Yuzheng Zhang, Yingxue |
| contents | Recent advancements in Large Language Models (LLMs) and Vision-Language Models (VLMs) have made them powerful tools in embodied navigation, enabling agents to leverage commonsense and spatial reasoning for efficient exploration in unfamiliar environments. Existing LLM-based approaches convert global memory, such as semantic or topological maps, into language descriptions to guide navigation. While this improves efficiency and reduces redundant exploration, the loss of geometric information in language-based representations hinders spatial reasoning, especially in intricate environments. To address this, VLM-based approaches directly process ego-centric visual inputs to select optimal directions for exploration. However, relying solely on a first-person perspective makes navigation a partially observed decision-making problem, leading to suboptimal decisions in complex environments. In this paper, we present a novel vision-language model (VLM)-based navigation framework that addresses these challenges by adaptively retrieving task-relevant cues from a global memory module and integrating them with the agent's egocentric observations. By dynamically aligning global contextual information with local perception, our approach enhances spatial reasoning and decision-making in long-horizon tasks. Experimental results demonstrate that the proposed method surpasses previous state-of-the-art approaches in object navigation tasks, providing a more effective and scalable solution for embodied navigation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_14254 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigation Zhang, Lingfeng Liu, Yuecheng Zhang, Zhanguang Aghaei, Matin Hu, Yaochen Gu, Hongjian Alomrani, Mohammad Ali Bravo, David Gamaliel Arcos Karimi, Raika Hamidizadeh, Atia Xu, Haoping Huang, Guowei Zhang, Zhanpeng Cao, Tongtong Qiu, Weichao Quan, Xingyue Hao, Jianye Zhuang, Yuzheng Zhang, Yingxue Robotics Artificial Intelligence Recent advancements in Large Language Models (LLMs) and Vision-Language Models (VLMs) have made them powerful tools in embodied navigation, enabling agents to leverage commonsense and spatial reasoning for efficient exploration in unfamiliar environments. Existing LLM-based approaches convert global memory, such as semantic or topological maps, into language descriptions to guide navigation. While this improves efficiency and reduces redundant exploration, the loss of geometric information in language-based representations hinders spatial reasoning, especially in intricate environments. To address this, VLM-based approaches directly process ego-centric visual inputs to select optimal directions for exploration. However, relying solely on a first-person perspective makes navigation a partially observed decision-making problem, leading to suboptimal decisions in complex environments. In this paper, we present a novel vision-language model (VLM)-based navigation framework that addresses these challenges by adaptively retrieving task-relevant cues from a global memory module and integrating them with the agent's egocentric observations. By dynamically aligning global contextual information with local perception, our approach enhances spatial reasoning and decision-making in long-horizon tasks. Experimental results demonstrate that the proposed method surpasses previous state-of-the-art approaches in object navigation tasks, providing a more effective and scalable solution for embodied navigation. |
| title | Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigation |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2502.14254 |