VISC: mmWave Radar Scene Flow Estimation using Pervasive Visual-Inertial Supervision
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916826668072960 |
|---|---|
| author | Liu, Kezhong Zhou, Yiwen Chen, Mozi He, Jianhua Xu, Jingao Yang, Zheng Lu, Chris Xiaoxuan Zhang, Shengkai |
| author_facet | Liu, Kezhong Zhou, Yiwen Chen, Mozi He, Jianhua Xu, Jingao Yang, Zheng Lu, Chris Xiaoxuan Zhang, Shengkai |
| contents | This work proposes a mmWave radar's scene flow estimation framework supervised by data from a widespread visual-inertial (VI) sensor suite, allowing crowdsourced training data from smart vehicles. Current scene flow estimation methods for mmWave radar are typically supervised by dense point clouds from 3D LiDARs, which are expensive and not widely available in smart vehicles. While VI data are more accessible, visual images alone cannot capture the 3D motions of moving objects, making it difficult to supervise their scene flow. Moreover, the temporal drift of VI rigid transformation also degenerates the scene flow estimation of static points. To address these challenges, we propose a drift-free rigid transformation estimator that fuses kinematic model-based ego-motions with neural network-learned results. It provides strong supervision signals to radar-based rigid transformation and infers the scene flow of static points. Then, we develop an optical-mmWave supervision extraction module that extracts the supervision signals of radar rigid transformation and scene flow. It strengthens the supervision by learning the scene flow of dynamic points with the joint constraints of optical and mmWave radar measurements. Extensive experiments demonstrate that, in smoke-filled environments, our method even outperforms state-of-the-art (SOTA) approaches using costly LiDARs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_03938 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VISC: mmWave Radar Scene Flow Estimation using Pervasive Visual-Inertial Supervision Liu, Kezhong Zhou, Yiwen Chen, Mozi He, Jianhua Xu, Jingao Yang, Zheng Lu, Chris Xiaoxuan Zhang, Shengkai Computer Vision and Pattern Recognition Robotics This work proposes a mmWave radar's scene flow estimation framework supervised by data from a widespread visual-inertial (VI) sensor suite, allowing crowdsourced training data from smart vehicles. Current scene flow estimation methods for mmWave radar are typically supervised by dense point clouds from 3D LiDARs, which are expensive and not widely available in smart vehicles. While VI data are more accessible, visual images alone cannot capture the 3D motions of moving objects, making it difficult to supervise their scene flow. Moreover, the temporal drift of VI rigid transformation also degenerates the scene flow estimation of static points. To address these challenges, we propose a drift-free rigid transformation estimator that fuses kinematic model-based ego-motions with neural network-learned results. It provides strong supervision signals to radar-based rigid transformation and infers the scene flow of static points. Then, we develop an optical-mmWave supervision extraction module that extracts the supervision signals of radar rigid transformation and scene flow. It strengthens the supervision by learning the scene flow of dynamic points with the joint constraints of optical and mmWave radar measurements. Extensive experiments demonstrate that, in smoke-filled environments, our method even outperforms state-of-the-art (SOTA) approaches using costly LiDARs. |
| title | VISC: mmWave Radar Scene Flow Estimation using Pervasive Visual-Inertial Supervision |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2507.03938 |