MonoTAKD: Teaching Assistant Knowledge Distillation for Monocular 3D Object Detection
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916662475751424 |
|---|---|
| author | Liu, Hou-I Wu, Christine Cheng, Jen-Hao Chai, Wenhao Wang, Shian-Yun Liu, Gaowen Latapie, Hugo Wu, Jhih-Ciang Hwang, Jenq-Neng Shuai, Hong-Han Cheng, Wen-Huang |
| author_facet | Liu, Hou-I Wu, Christine Cheng, Jen-Hao Chai, Wenhao Wang, Shian-Yun Liu, Gaowen Latapie, Hugo Wu, Jhih-Ciang Hwang, Jenq-Neng Shuai, Hong-Han Cheng, Wen-Huang |
| contents | Monocular 3D object detection (Mono3D) holds noteworthy promise for autonomous driving applications owing to the cost-effectiveness and rich visual context of monocular camera sensors. However, depth ambiguity poses a significant challenge, as it requires extracting precise 3D scene geometry from a single image, resulting in suboptimal performance when transferring knowledge from a LiDAR-based teacher model to a camera-based student model. To facilitate effective distillation, we introduce Monocular Teaching Assistant Knowledge Distillation (MonoTAKD), which proposes a camera-based teaching assistant (TA) model to transfer robust 3D visual knowledge to the student model, leveraging the smaller feature representation gap. Additionally, we define 3D spatial cues as residual features that capture the differences between the teacher and the TA models. We then leverage these cues to improve the student model's 3D perception capabilities. Experimental results show that our MonoTAKD achieves state-of-the-art performance on the KITTI3D dataset. Furthermore, we evaluate the performance on nuScenes and KITTI raw datasets to demonstrate the generalization of our model to multi-view 3D and unsupervised data settings. Our code is available at https://github.com/hoiliu-0801/MonoTAKD. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_04910 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MonoTAKD: Teaching Assistant Knowledge Distillation for Monocular 3D Object Detection Liu, Hou-I Wu, Christine Cheng, Jen-Hao Chai, Wenhao Wang, Shian-Yun Liu, Gaowen Latapie, Hugo Wu, Jhih-Ciang Hwang, Jenq-Neng Shuai, Hong-Han Cheng, Wen-Huang Computer Vision and Pattern Recognition Monocular 3D object detection (Mono3D) holds noteworthy promise for autonomous driving applications owing to the cost-effectiveness and rich visual context of monocular camera sensors. However, depth ambiguity poses a significant challenge, as it requires extracting precise 3D scene geometry from a single image, resulting in suboptimal performance when transferring knowledge from a LiDAR-based teacher model to a camera-based student model. To facilitate effective distillation, we introduce Monocular Teaching Assistant Knowledge Distillation (MonoTAKD), which proposes a camera-based teaching assistant (TA) model to transfer robust 3D visual knowledge to the student model, leveraging the smaller feature representation gap. Additionally, we define 3D spatial cues as residual features that capture the differences between the teacher and the TA models. We then leverage these cues to improve the student model's 3D perception capabilities. Experimental results show that our MonoTAKD achieves state-of-the-art performance on the KITTI3D dataset. Furthermore, we evaluate the performance on nuScenes and KITTI raw datasets to demonstrate the generalization of our model to multi-view 3D and unsupervised data settings. Our code is available at https://github.com/hoiliu-0801/MonoTAKD. |
| title | MonoTAKD: Teaching Assistant Knowledge Distillation for Monocular 3D Object Detection |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2404.04910 |