Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.01889 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912740966137856 |
|---|---|
| author | Nasser, Zaid Iumanov, Mikhail Li, Tianhao Popov, Maxim Mahmoud, Jaafar Mohrat, Malik Obrubov, Ilya Derevyanka, Ekaterina Sosin, Ivan Kolyubin, Sergey |
| author_facet | Nasser, Zaid Iumanov, Mikhail Li, Tianhao Popov, Maxim Mahmoud, Jaafar Mohrat, Malik Obrubov, Ilya Derevyanka, Ekaterina Sosin, Ivan Kolyubin, Sergey |
| contents | We present KM-ViPE (Knowledge Mapping Video Pose Engine), a real-time open-vocabulary SLAM framework for uncalibrated monocular cameras in dynamic environments. Unlike systems requiring depth sensors and offline calibration, KM-ViPE operates directly on raw RGB streams, making it ideal for ego-centric applications and harvesting internet-scale video data for training. KM-ViPE tightly couples DINO visual features with geometric constraints through a high-level features based adaptive robust kernel that handles both moving objects and movable static objects (e.g., moving furniture in ego-centric views). The system performs simultaneous online localization and open-vocabulary semantic mapping by fusing geometric and deep visual features aligned with language embeddings. Our results are competitive with state-of-the-art approaches, while existing solutions either operate offline, need depth data and/or odometry estimation, or lack dynamic scene robustness. KM-ViPE benefits from internet-scale training and uniquely combines online operation, uncalibrated monocular input, and robust handling of dynamic scenes, which makes it a good fit for autonomous robotics and AR/VR applications and advances practical spatial intelligence capabilities for embodied AI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_01889 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | KM-ViPE: Online Tightly Coupled Vision-Language-Geometry Fusion for Open-Vocabulary Semantic SLAM Nasser, Zaid Iumanov, Mikhail Li, Tianhao Popov, Maxim Mahmoud, Jaafar Mohrat, Malik Obrubov, Ilya Derevyanka, Ekaterina Sosin, Ivan Kolyubin, Sergey Computer Vision and Pattern Recognition We present KM-ViPE (Knowledge Mapping Video Pose Engine), a real-time open-vocabulary SLAM framework for uncalibrated monocular cameras in dynamic environments. Unlike systems requiring depth sensors and offline calibration, KM-ViPE operates directly on raw RGB streams, making it ideal for ego-centric applications and harvesting internet-scale video data for training. KM-ViPE tightly couples DINO visual features with geometric constraints through a high-level features based adaptive robust kernel that handles both moving objects and movable static objects (e.g., moving furniture in ego-centric views). The system performs simultaneous online localization and open-vocabulary semantic mapping by fusing geometric and deep visual features aligned with language embeddings. Our results are competitive with state-of-the-art approaches, while existing solutions either operate offline, need depth data and/or odometry estimation, or lack dynamic scene robustness. KM-ViPE benefits from internet-scale training and uniquely combines online operation, uncalibrated monocular input, and robust handling of dynamic scenes, which makes it a good fit for autonomous robotics and AR/VR applications and advances practical spatial intelligence capabilities for embodied AI. |
| title | KM-ViPE: Online Tightly Coupled Vision-Language-Geometry Fusion for Open-Vocabulary Semantic SLAM |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.01889 |