Gaussian Splatting on the Move: Blur and Rolling Shutter Compensation for Natural Camera Motion
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910529959755776 |
|---|---|
| author | Seiskari, Otto Ylilammi, Jerry Kaatrasalo, Valtteri Rantalankila, Pekka Turkulainen, Matias Kannala, Juho Rahtu, Esa Solin, Arno |
| author_facet | Seiskari, Otto Ylilammi, Jerry Kaatrasalo, Valtteri Rantalankila, Pekka Turkulainen, Matias Kannala, Juho Rahtu, Esa Solin, Arno |
| contents | High-quality scene reconstruction and novel view synthesis based on Gaussian Splatting (3DGS) typically require steady, high-quality photographs, often impractical to capture with handheld cameras. We present a method that adapts to camera motion and allows high-quality scene reconstruction with handheld video data suffering from motion blur and rolling shutter distortion. Our approach is based on detailed modelling of the physical image formation process and utilizes velocities estimated using visual-inertial odometry (VIO). Camera poses are considered non-static during the exposure time of a single image frame and camera poses are further optimized in the reconstruction process. We formulate a differentiable rendering pipeline that leverages screen space approximation to efficiently incorporate rolling-shutter and motion blur effects into the 3DGS framework. Our results with both synthetic and real data demonstrate superior performance in mitigating camera motion over existing methods, thereby advancing 3DGS in naturalistic settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_13327 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Gaussian Splatting on the Move: Blur and Rolling Shutter Compensation for Natural Camera Motion Seiskari, Otto Ylilammi, Jerry Kaatrasalo, Valtteri Rantalankila, Pekka Turkulainen, Matias Kannala, Juho Rahtu, Esa Solin, Arno Computer Vision and Pattern Recognition High-quality scene reconstruction and novel view synthesis based on Gaussian Splatting (3DGS) typically require steady, high-quality photographs, often impractical to capture with handheld cameras. We present a method that adapts to camera motion and allows high-quality scene reconstruction with handheld video data suffering from motion blur and rolling shutter distortion. Our approach is based on detailed modelling of the physical image formation process and utilizes velocities estimated using visual-inertial odometry (VIO). Camera poses are considered non-static during the exposure time of a single image frame and camera poses are further optimized in the reconstruction process. We formulate a differentiable rendering pipeline that leverages screen space approximation to efficiently incorporate rolling-shutter and motion blur effects into the 3DGS framework. Our results with both synthetic and real data demonstrate superior performance in mitigating camera motion over existing methods, thereby advancing 3DGS in naturalistic settings. |
| title | Gaussian Splatting on the Move: Blur and Rolling Shutter Compensation for Natural Camera Motion |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.13327 |