Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2505.01857 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910927311339520 |
|---|---|
| author | Li, Haoteng Yang, Zhao Qian, Zezhong Zhao, Gongpeng Huang, Yuqi Yu, Jun Zhou, Huazheng Liu, Longjun |
| author_facet | Li, Haoteng Yang, Zhao Qian, Zezhong Zhao, Gongpeng Huang, Yuqi Yu, Jun Zhou, Huazheng Liu, Longjun |
| contents | Accurate and high-fidelity driving scene reconstruction relies on fully leveraging scene information as conditioning. However, existing approaches, which primarily use 3D bounding boxes and binary maps for foreground and background control, fall short in capturing the complexity of the scene and integrating multi-modal information. In this paper, we propose DualDiff, a dual-branch conditional diffusion model designed to enhance multi-view driving scene generation. We introduce Occupancy Ray Sampling (ORS), a semantic-rich 3D representation, alongside numerical driving scene representation, for comprehensive foreground and background control. To improve cross-modal information integration, we propose a Semantic Fusion Attention (SFA) mechanism that aligns and fuses features across modalities. Furthermore, we design a foreground-aware masked (FGM) loss to enhance the generation of tiny objects. DualDiff achieves state-of-the-art performance in FID score, as well as consistently better results in downstream BEV segmentation and 3D object detection tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_01857 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DualDiff: Dual-branch Diffusion Model for Autonomous Driving with Semantic Fusion Li, Haoteng Yang, Zhao Qian, Zezhong Zhao, Gongpeng Huang, Yuqi Yu, Jun Zhou, Huazheng Liu, Longjun Computer Vision and Pattern Recognition Accurate and high-fidelity driving scene reconstruction relies on fully leveraging scene information as conditioning. However, existing approaches, which primarily use 3D bounding boxes and binary maps for foreground and background control, fall short in capturing the complexity of the scene and integrating multi-modal information. In this paper, we propose DualDiff, a dual-branch conditional diffusion model designed to enhance multi-view driving scene generation. We introduce Occupancy Ray Sampling (ORS), a semantic-rich 3D representation, alongside numerical driving scene representation, for comprehensive foreground and background control. To improve cross-modal information integration, we propose a Semantic Fusion Attention (SFA) mechanism that aligns and fuses features across modalities. Furthermore, we design a foreground-aware masked (FGM) loss to enhance the generation of tiny objects. DualDiff achieves state-of-the-art performance in FID score, as well as consistently better results in downstream BEV segmentation and 3D object detection tasks. |
| title | DualDiff: Dual-branch Diffusion Model for Autonomous Driving with Semantic Fusion |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.01857 |