Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.06682 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915222673948672 |
|---|---|
| author | Pan, Tai-Yu Jeon, Sooyoung Fan, Mengdi Yoo, Jinsu Feng, Zhenyang Campbell, Mark Weinberger, Kilian Q. Hariharan, Bharath Chao, Wei-Lun |
| author_facet | Pan, Tai-Yu Jeon, Sooyoung Fan, Mengdi Yoo, Jinsu Feng, Zhenyang Campbell, Mark Weinberger, Kilian Q. Hariharan, Bharath Chao, Wei-Lun |
| contents | Self-driving cars relying solely on ego-centric perception face limitations in sensing, often failing to detect occluded, faraway objects. Collaborative autonomous driving (CAV) seems like a promising direction, but collecting data for development is non-trivial. It requires placing multiple sensor-equipped agents in a real-world driving scene, simultaneously! As such, existing datasets are limited in locations and agents. We introduce a novel surrogate to the rescue, which is to generate realistic perception from different viewpoints in a driving scene, conditioned on a real-world sample - the ego-car's sensory data. This surrogate has huge potential: it could potentially turn any ego-car dataset into a collaborative driving one to scale up the development of CAV. We present the very first solution, using a combination of simulated collaborative data and real ego-car data. Our method, Transfer Your Perspective (TYP), learns a conditioned diffusion model whose output samples are not only realistic but also consistent in both semantics and layouts with the given ego-car data. Empirical results demonstrate TYP's effectiveness in aiding in a CAV setting. In particular, TYP enables us to (pre-)train collaborative perception algorithms like early and late fusion with little or no real-world collaborative data, greatly facilitating downstream CAV applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_06682 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Transfer Your Perspective: Controllable 3D Generation from Any Viewpoint in a Driving Scene Pan, Tai-Yu Jeon, Sooyoung Fan, Mengdi Yoo, Jinsu Feng, Zhenyang Campbell, Mark Weinberger, Kilian Q. Hariharan, Bharath Chao, Wei-Lun Computer Vision and Pattern Recognition Self-driving cars relying solely on ego-centric perception face limitations in sensing, often failing to detect occluded, faraway objects. Collaborative autonomous driving (CAV) seems like a promising direction, but collecting data for development is non-trivial. It requires placing multiple sensor-equipped agents in a real-world driving scene, simultaneously! As such, existing datasets are limited in locations and agents. We introduce a novel surrogate to the rescue, which is to generate realistic perception from different viewpoints in a driving scene, conditioned on a real-world sample - the ego-car's sensory data. This surrogate has huge potential: it could potentially turn any ego-car dataset into a collaborative driving one to scale up the development of CAV. We present the very first solution, using a combination of simulated collaborative data and real ego-car data. Our method, Transfer Your Perspective (TYP), learns a conditioned diffusion model whose output samples are not only realistic but also consistent in both semantics and layouts with the given ego-car data. Empirical results demonstrate TYP's effectiveness in aiding in a CAV setting. In particular, TYP enables us to (pre-)train collaborative perception algorithms like early and late fusion with little or no real-world collaborative data, greatly facilitating downstream CAV applications. |
| title | Transfer Your Perspective: Controllable 3D Generation from Any Viewpoint in a Driving Scene |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2502.06682 |