Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2601.21288 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917231084961792 |
|---|---|
| author | Lian, Weitong Tang, Zecong Li, Haoran Gao, Tianjian Wang, Yifei Wang, Zixu Meng, Lingyi Ru, Tengju Cui, Zhejun Zhu, Yichen Cao, Hangshuo Kang, Qi Chen, Tianxing Qin, Yusen Wang, Kaixuan Zhang, Yu |
| author_facet | Lian, Weitong Tang, Zecong Li, Haoran Gao, Tianjian Wang, Yifei Wang, Zixu Meng, Lingyi Ru, Tengju Cui, Zhejun Zhu, Yichen Cao, Hangshuo Kang, Qi Chen, Tianxing Qin, Yusen Wang, Kaixuan Zhang, Yu |
| contents | Autonomous driving is an important and safety-critical task, and recent advances in LLMs/VLMs have opened new possibilities for reasoning and planning in this domain. However, large models demand substantial GPU memory and exhibit high inference latency, while conventional supervised fine-tuning (SFT) often struggles to bridge the capability gaps of small models. To address these limitations, we propose Drive-KD, a framework that decomposes autonomous driving into a "perception-reasoning-planning" triad and transfers these capabilities via knowledge distillation. We identify layer-specific attention as the distillation signal to construct capability-specific single-teacher models that outperform baselines. Moreover, we unify these single-teacher settings into a multi-teacher distillation framework and introduce asymmetric gradient projection to mitigate cross-capability gradient conflicts. Extensive evaluations validate the generalization of our method across diverse model families and scales. Experiments show that our distilled InternVL3-1B model, with ~42 times less GPU memory and ~11.4 times higher throughput, achieves better overall performance than the pretrained 78B model from the same family on DriveBench, and surpasses GPT-5.1 on the planning dimension, providing insights toward efficient autonomous driving VLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_21288 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Drive-KD: Multi-Teacher Distillation for VLMs in Autonomous Driving Lian, Weitong Tang, Zecong Li, Haoran Gao, Tianjian Wang, Yifei Wang, Zixu Meng, Lingyi Ru, Tengju Cui, Zhejun Zhu, Yichen Cao, Hangshuo Kang, Qi Chen, Tianxing Qin, Yusen Wang, Kaixuan Zhang, Yu Artificial Intelligence Computer Vision and Pattern Recognition Autonomous driving is an important and safety-critical task, and recent advances in LLMs/VLMs have opened new possibilities for reasoning and planning in this domain. However, large models demand substantial GPU memory and exhibit high inference latency, while conventional supervised fine-tuning (SFT) often struggles to bridge the capability gaps of small models. To address these limitations, we propose Drive-KD, a framework that decomposes autonomous driving into a "perception-reasoning-planning" triad and transfers these capabilities via knowledge distillation. We identify layer-specific attention as the distillation signal to construct capability-specific single-teacher models that outperform baselines. Moreover, we unify these single-teacher settings into a multi-teacher distillation framework and introduce asymmetric gradient projection to mitigate cross-capability gradient conflicts. Extensive evaluations validate the generalization of our method across diverse model families and scales. Experiments show that our distilled InternVL3-1B model, with ~42 times less GPU memory and ~11.4 times higher throughput, achieves better overall performance than the pretrained 78B model from the same family on DriveBench, and surpasses GPT-5.1 on the planning dimension, providing insights toward efficient autonomous driving VLMs. |
| title | Drive-KD: Multi-Teacher Distillation for VLMs in Autonomous Driving |
| topic | Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2601.21288 |