SpaAct: Spatially-Activated Transition Learning with Curriculum Adaptation for Vision-Language Navigation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909004031066112 |
|---|---|
| author | Li, Pengna Wu, Kangyi Xu, Shaoqing Li, Fang Li, Hanbing Zhao, Lin Lyu, Kailin Chen, Long Yang, Zhi-Xin Zheng, Nanning |
| author_facet | Li, Pengna Wu, Kangyi Xu, Shaoqing Li, Fang Li, Hanbing Zhao, Lin Lyu, Kailin Chen, Long Yang, Zhi-Xin Zheng, Nanning |
| contents | Vision-and-Language Navigation (VLN) aims to enable an embodied agent to follow natural-language instructions and navigate to a target location in unseen 3D environments. We argue that adapting VLMs to VLN requires endowing them with two complementary capabilities for acquiring such awareness, namely backward action reasoning (why) and forward transition prediction~(how). Based on this insight, we propose SpaAct, a simple yet effective training framework that activates the dynamic spatial awareness in VLMs. Specifically, SpaAct introduces two spatial activation tasks: Action Retrospection, which asks the model to infer the executed action sequence from visual transitions, and Future Frame Selection, which forces the model to predict the visual transitions conditioned on history and action. These two objectives provide lightweight supervision on both backward action reasoning and forward transition prediction, encouraging the model to build dynamic spatial awareness in a VLM-friendly way. To further stabilize adaptation, we design TriPA, a Tri-factor Progressive Adaptive curriculum learning method that organizes training samples from easy to hard, allowing the model to gradually acquire navigation skills from basic locomotion to long-horizon reasoning. Experiments on standard VLN-CE benchmarks show that SpaAct consistently improves VLM-based navigation and achieves state-of-the-art performance. We will release the code and models to support future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_27620 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SpaAct: Spatially-Activated Transition Learning with Curriculum Adaptation for Vision-Language Navigation Li, Pengna Wu, Kangyi Xu, Shaoqing Li, Fang Li, Hanbing Zhao, Lin Lyu, Kailin Chen, Long Yang, Zhi-Xin Zheng, Nanning Computer Vision and Pattern Recognition Vision-and-Language Navigation (VLN) aims to enable an embodied agent to follow natural-language instructions and navigate to a target location in unseen 3D environments. We argue that adapting VLMs to VLN requires endowing them with two complementary capabilities for acquiring such awareness, namely backward action reasoning (why) and forward transition prediction~(how). Based on this insight, we propose SpaAct, a simple yet effective training framework that activates the dynamic spatial awareness in VLMs. Specifically, SpaAct introduces two spatial activation tasks: Action Retrospection, which asks the model to infer the executed action sequence from visual transitions, and Future Frame Selection, which forces the model to predict the visual transitions conditioned on history and action. These two objectives provide lightweight supervision on both backward action reasoning and forward transition prediction, encouraging the model to build dynamic spatial awareness in a VLM-friendly way. To further stabilize adaptation, we design TriPA, a Tri-factor Progressive Adaptive curriculum learning method that organizes training samples from easy to hard, allowing the model to gradually acquire navigation skills from basic locomotion to long-horizon reasoning. Experiments on standard VLN-CE benchmarks show that SpaAct consistently improves VLM-based navigation and achieves state-of-the-art performance. We will release the code and models to support future research. |
| title | SpaAct: Spatially-Activated Transition Learning with Curriculum Adaptation for Vision-Language Navigation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2604.27620 |