LTDA-Drive: LLMs-guided Generative Models based Long-tail Data Augmentation for Autonomous Driving
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912391857438720 |
|---|---|
| author | Yurt, Mahmut Ye, Xin Ma, Yunsheng Luo, Jingru Mallik, Abhirup Pauly, John Yaman, Burhaneddin Ren, Liu |
| author_facet | Yurt, Mahmut Ye, Xin Ma, Yunsheng Luo, Jingru Mallik, Abhirup Pauly, John Yaman, Burhaneddin Ren, Liu |
| contents | 3D perception plays an essential role for improving the safety and performance of autonomous driving. Yet, existing models trained on real-world datasets, which naturally exhibit long-tail distributions, tend to underperform on rare and safety-critical, vulnerable classes, such as pedestrians and cyclists. Existing studies on reweighting and resampling techniques struggle with the scarcity and limited diversity within tail classes. To address these limitations, we introduce LTDA-Drive, a novel LLM-guided data augmentation framework designed to synthesize diverse, high-quality long-tail samples. LTDA-Drive replaces head-class objects in driving scenes with tail-class objects through a three-stage process: (1) text-guided diffusion models remove head-class objects, (2) generative models insert instances of the tail classes, and (3) an LLM agent filters out low-quality synthesized images. Experiments conducted on the KITTI dataset show that LTDA-Drive significantly improves tail-class detection, achieving 34.75\% improvement for rare classes over counterpart methods. These results further highlight the effectiveness of LTDA-Drive in tackling long-tail challenges by generating high-quality and diverse data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_18198 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LTDA-Drive: LLMs-guided Generative Models based Long-tail Data Augmentation for Autonomous Driving Yurt, Mahmut Ye, Xin Ma, Yunsheng Luo, Jingru Mallik, Abhirup Pauly, John Yaman, Burhaneddin Ren, Liu Robotics 3D perception plays an essential role for improving the safety and performance of autonomous driving. Yet, existing models trained on real-world datasets, which naturally exhibit long-tail distributions, tend to underperform on rare and safety-critical, vulnerable classes, such as pedestrians and cyclists. Existing studies on reweighting and resampling techniques struggle with the scarcity and limited diversity within tail classes. To address these limitations, we introduce LTDA-Drive, a novel LLM-guided data augmentation framework designed to synthesize diverse, high-quality long-tail samples. LTDA-Drive replaces head-class objects in driving scenes with tail-class objects through a three-stage process: (1) text-guided diffusion models remove head-class objects, (2) generative models insert instances of the tail classes, and (3) an LLM agent filters out low-quality synthesized images. Experiments conducted on the KITTI dataset show that LTDA-Drive significantly improves tail-class detection, achieving 34.75\% improvement for rare classes over counterpart methods. These results further highlight the effectiveness of LTDA-Drive in tackling long-tail challenges by generating high-quality and diverse data. |
| title | LTDA-Drive: LLMs-guided Generative Models based Long-tail Data Augmentation for Autonomous Driving |
| topic | Robotics |
| url | https://arxiv.org/abs/2505.18198 |