FreeSim: Toward Free-viewpoint Camera Simulation in Driving Scenes
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916507716419584 |
|---|---|
| author | Fan, Lue Zhang, Hao Wang, Qitai Li, Hongsheng Zhang, Zhaoxiang |
| author_facet | Fan, Lue Zhang, Hao Wang, Qitai Li, Hongsheng Zhang, Zhaoxiang |
| contents | We propose FreeSim, a camera simulation method for autonomous driving. FreeSim emphasizes high-quality rendering from viewpoints beyond the recorded ego trajectories. In such viewpoints, previous methods have unacceptable degradation because the training data of these viewpoints is unavailable. To address such data scarcity, we first propose a generative enhancement model with a matched data construction strategy. The resulting model can generate high-quality images in a viewpoint slightly deviated from the recorded trajectories, conditioned on the degraded rendering of this viewpoint. We then propose a progressive reconstruction strategy, which progressively adds generated images of unrecorded views into the reconstruction process, starting from slightly off-trajectory viewpoints and moving progressively farther away. With this progressive generation-reconstruction pipeline, FreeSim supports high-quality off-trajectory view synthesis under large deviations of more than 3 meters. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_03566 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | FreeSim: Toward Free-viewpoint Camera Simulation in Driving Scenes Fan, Lue Zhang, Hao Wang, Qitai Li, Hongsheng Zhang, Zhaoxiang Computer Vision and Pattern Recognition We propose FreeSim, a camera simulation method for autonomous driving. FreeSim emphasizes high-quality rendering from viewpoints beyond the recorded ego trajectories. In such viewpoints, previous methods have unacceptable degradation because the training data of these viewpoints is unavailable. To address such data scarcity, we first propose a generative enhancement model with a matched data construction strategy. The resulting model can generate high-quality images in a viewpoint slightly deviated from the recorded trajectories, conditioned on the degraded rendering of this viewpoint. We then propose a progressive reconstruction strategy, which progressively adds generated images of unrecorded views into the reconstruction process, starting from slightly off-trajectory viewpoints and moving progressively farther away. With this progressive generation-reconstruction pipeline, FreeSim supports high-quality off-trajectory view synthesis under large deviations of more than 3 meters. |
| title | FreeSim: Toward Free-viewpoint Camera Simulation in Driving Scenes |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.03566 |