ViLAM: Distilling Vision-Language Reasoning into Attention Maps for Social Robot Navigation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911494405357568 |
|---|---|
| author | Elnoor, Mohamed Weerakoon, Kasun Seneviratne, Gershom Liang, Jing Rajagopal, Vignesh Manocha, Dinesh |
| author_facet | Elnoor, Mohamed Weerakoon, Kasun Seneviratne, Gershom Liang, Jing Rajagopal, Vignesh Manocha, Dinesh |
| contents | We introduce ViLAM, a novel method for distilling vision-language reasoning from large Vision-Language Models (VLMs) into spatial attention maps for socially compliant robot navigation. Unlike traditional methods that rely on expert demonstrations or human-annotated datasets, ViLAM performs knowledge distillation and fine-tuning at the intermediate layer representation (attention) level by aligning attention maps from a pretrained vision-action model with socially guided attention maps derived from a large VLM. These distilled attention maps highlight key navigational regions in a scene and serve as socially informed spatial cost maps for motion planning. To achieve this, we introduce a novel attention-level distillation loss that fuses knowledge from both sources, generating augmented attention maps with enhanced social awareness. These refined attention maps are then used as a traversability costmap within a socially aware local planner for navigation. We validate our approach through real-world experiments on a Husky wheeled robot, and demonstrate 14.2% - 50% improvements in success rate over existing methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_09820 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ViLAM: Distilling Vision-Language Reasoning into Attention Maps for Social Robot Navigation Elnoor, Mohamed Weerakoon, Kasun Seneviratne, Gershom Liang, Jing Rajagopal, Vignesh Manocha, Dinesh Robotics Artificial Intelligence We introduce ViLAM, a novel method for distilling vision-language reasoning from large Vision-Language Models (VLMs) into spatial attention maps for socially compliant robot navigation. Unlike traditional methods that rely on expert demonstrations or human-annotated datasets, ViLAM performs knowledge distillation and fine-tuning at the intermediate layer representation (attention) level by aligning attention maps from a pretrained vision-action model with socially guided attention maps derived from a large VLM. These distilled attention maps highlight key navigational regions in a scene and serve as socially informed spatial cost maps for motion planning. To achieve this, we introduce a novel attention-level distillation loss that fuses knowledge from both sources, generating augmented attention maps with enhanced social awareness. These refined attention maps are then used as a traversability costmap within a socially aware local planner for navigation. We validate our approach through real-world experiments on a Husky wheeled robot, and demonstrate 14.2% - 50% improvements in success rate over existing methods. |
| title | ViLAM: Distilling Vision-Language Reasoning into Attention Maps for Social Robot Navigation |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2503.09820 |