GASP: Gaussian Avatars with Synthetic Priors
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915057234870272 |
|---|---|
| author | Saunders, Jack Hewitt, Charlie Jian, Yanan Kowalski, Marek Baltrusaitis, Tadas Chen, Yiye Cosker, Darren Estellers, Virginia Gyde, Nicholas Namboodiri, Vinay P. Lundell, Benjamin E |
| author_facet | Saunders, Jack Hewitt, Charlie Jian, Yanan Kowalski, Marek Baltrusaitis, Tadas Chen, Yiye Cosker, Darren Estellers, Virginia Gyde, Nicholas Namboodiri, Vinay P. Lundell, Benjamin E |
| contents | Gaussian Splatting has changed the game for real-time photo-realistic rendering. One of the most popular applications of Gaussian Splatting is to create animatable avatars, known as Gaussian Avatars. Recent works have pushed the boundaries of quality and rendering efficiency but suffer from two main limitations. Either they require expensive multi-camera rigs to produce avatars with free-view rendering, or they can be trained with a single camera but only rendered at high quality from this fixed viewpoint. An ideal model would be trained using a short monocular video or image from available hardware, such as a webcam, and rendered from any view. To this end, we propose GASP: Gaussian Avatars with Synthetic Priors. To overcome the limitations of existing datasets, we exploit the pixel-perfect nature of synthetic data to train a Gaussian Avatar prior. By fitting this prior model to a single photo or video and fine-tuning it, we get a high-quality Gaussian Avatar, which supports 360$^\circ$ rendering. Our prior is only required for fitting, not inference, enabling real-time application. Through our method, we obtain high-quality, animatable Avatars from limited data which can be animated and rendered at 70fps on commercial hardware. See our project page (https://microsoft.github.io/GASP/) for results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_07739 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | GASP: Gaussian Avatars with Synthetic Priors Saunders, Jack Hewitt, Charlie Jian, Yanan Kowalski, Marek Baltrusaitis, Tadas Chen, Yiye Cosker, Darren Estellers, Virginia Gyde, Nicholas Namboodiri, Vinay P. Lundell, Benjamin E Computer Vision and Pattern Recognition Artificial Intelligence Graphics Gaussian Splatting has changed the game for real-time photo-realistic rendering. One of the most popular applications of Gaussian Splatting is to create animatable avatars, known as Gaussian Avatars. Recent works have pushed the boundaries of quality and rendering efficiency but suffer from two main limitations. Either they require expensive multi-camera rigs to produce avatars with free-view rendering, or they can be trained with a single camera but only rendered at high quality from this fixed viewpoint. An ideal model would be trained using a short monocular video or image from available hardware, such as a webcam, and rendered from any view. To this end, we propose GASP: Gaussian Avatars with Synthetic Priors. To overcome the limitations of existing datasets, we exploit the pixel-perfect nature of synthetic data to train a Gaussian Avatar prior. By fitting this prior model to a single photo or video and fine-tuning it, we get a high-quality Gaussian Avatar, which supports 360$^\circ$ rendering. Our prior is only required for fitting, not inference, enabling real-time application. Through our method, we obtain high-quality, animatable Avatars from limited data which can be animated and rendered at 70fps on commercial hardware. See our project page (https://microsoft.github.io/GASP/) for results. |
| title | GASP: Gaussian Avatars with Synthetic Priors |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Graphics |
| url | https://arxiv.org/abs/2412.07739 |