MapDream: Task-Driven Map Learning for Vision-Language Navigation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914303085379584 |
|---|---|
| author | Lian, Guoxin Wang, Shuo Wang, Yucheng Wang, Yongcai Chen, Maiyue Wang, Kaihui Zhang, Bo Su, Zhizhong Li, Deying Fan, Zhaoxin |
| author_facet | Lian, Guoxin Wang, Shuo Wang, Yucheng Wang, Yongcai Chen, Maiyue Wang, Kaihui Zhang, Bo Su, Zhizhong Li, Deying Fan, Zhaoxin |
| contents | Vision-Language Navigation (VLN) requires agents to follow natural language instructions in partially observed 3D environments, motivating map representations that aggregate spatial context beyond local perception. However, most existing approaches rely on hand-crafted maps constructed independently of the navigation policy. We argue that maps should instead be learned representations shaped directly by navigation objectives rather than exhaustive reconstructions. Based on this insight, we propose MapDream, a map-in-the-loop framework that formulates map construction as autoregressive bird's-eye-view (BEV) image synthesis. The framework jointly learns map generation and action prediction, distilling environmental context into a compact three-channel BEV map that preserves only navigation-critical affordances. Supervised pre-training bootstraps a reliable mapping-to-control interface, while the autoregressive design enables end-to-end joint optimization through reinforcement fine-tuning. Experiments on R2R-CE and RxR-CE achieve state-of-the-art monocular performance, validating task-driven generative map learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_00222 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | MapDream: Task-Driven Map Learning for Vision-Language Navigation Lian, Guoxin Wang, Shuo Wang, Yucheng Wang, Yongcai Chen, Maiyue Wang, Kaihui Zhang, Bo Su, Zhizhong Li, Deying Fan, Zhaoxin Robotics Artificial Intelligence Computer Vision and Pattern Recognition Vision-Language Navigation (VLN) requires agents to follow natural language instructions in partially observed 3D environments, motivating map representations that aggregate spatial context beyond local perception. However, most existing approaches rely on hand-crafted maps constructed independently of the navigation policy. We argue that maps should instead be learned representations shaped directly by navigation objectives rather than exhaustive reconstructions. Based on this insight, we propose MapDream, a map-in-the-loop framework that formulates map construction as autoregressive bird's-eye-view (BEV) image synthesis. The framework jointly learns map generation and action prediction, distilling environmental context into a compact three-channel BEV map that preserves only navigation-critical affordances. Supervised pre-training bootstraps a reliable mapping-to-control interface, while the autoregressive design enables end-to-end joint optimization through reinforcement fine-tuning. Experiments on R2R-CE and RxR-CE achieve state-of-the-art monocular performance, validating task-driven generative map learning. |
| title | MapDream: Task-Driven Map Learning for Vision-Language Navigation |
| topic | Robotics Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2602.00222 |