LISN: Language-Instructed Social Navigation with VLM-based Controller Modulating
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908704493797376 |
|---|---|
| author | Chen, Junting Li, Yunchuan Jiang, Panfeng Du, Jiacheng Chen, Zixuan Tie, Chenrui Deng, Jiajun Shao, Lin |
| author_facet | Chen, Junting Li, Yunchuan Jiang, Panfeng Du, Jiacheng Chen, Zixuan Tie, Chenrui Deng, Jiajun Shao, Lin |
| contents | Towards human-robot coexistence, socially aware navigation is significant for mobile robots. Yet existing studies on this area focus mainly on path efficiency and pedestrian collision avoidance, which are essential but represent only a fraction of social navigation. Beyond these basics, robots must also comply with user instructions, aligning their actions to task goals and social norms expressed by humans. In this work, we present LISN-Bench, the first simulation-based benchmark for language-instructed social navigation. Built on Rosnav-Arena 3.0, it is the first standardized social navigation benchmark to incorporate instruction following and scene understanding across diverse contexts. To address this task, we further propose Social-Nav-Modulator, a fast-slow hierarchical system where a VLM agent modulates costmaps and controller parameters. Decoupling low-level action generation from the slower VLM loop reduces reliance on high-frequency VLM inference while improving dynamic avoidance and perception adaptability. Our method achieves an average success rate of 91.3%, which is greater than 63% than the most competitive baseline, with most of the improvements observed in challenging tasks such as following a person in a crowd and navigating while strictly avoiding instruction-forbidden regions. The project website is at: https://social-nav.github.io/LISN-project/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_09920 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LISN: Language-Instructed Social Navigation with VLM-based Controller Modulating Chen, Junting Li, Yunchuan Jiang, Panfeng Du, Jiacheng Chen, Zixuan Tie, Chenrui Deng, Jiajun Shao, Lin Robotics Artificial Intelligence Computer Vision and Pattern Recognition I.2.9 Towards human-robot coexistence, socially aware navigation is significant for mobile robots. Yet existing studies on this area focus mainly on path efficiency and pedestrian collision avoidance, which are essential but represent only a fraction of social navigation. Beyond these basics, robots must also comply with user instructions, aligning their actions to task goals and social norms expressed by humans. In this work, we present LISN-Bench, the first simulation-based benchmark for language-instructed social navigation. Built on Rosnav-Arena 3.0, it is the first standardized social navigation benchmark to incorporate instruction following and scene understanding across diverse contexts. To address this task, we further propose Social-Nav-Modulator, a fast-slow hierarchical system where a VLM agent modulates costmaps and controller parameters. Decoupling low-level action generation from the slower VLM loop reduces reliance on high-frequency VLM inference while improving dynamic avoidance and perception adaptability. Our method achieves an average success rate of 91.3%, which is greater than 63% than the most competitive baseline, with most of the improvements observed in challenging tasks such as following a person in a crowd and navigating while strictly avoiding instruction-forbidden regions. The project website is at: https://social-nav.github.io/LISN-project/ |
| title | LISN: Language-Instructed Social Navigation with VLM-based Controller Modulating |
| topic | Robotics Artificial Intelligence Computer Vision and Pattern Recognition I.2.9 |
| url | https://arxiv.org/abs/2512.09920 |