Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2504.09671 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909578762911744 |
|---|---|
| author | Manu, Pranav Srivastava, Astitva Raj, Amit Jampani, Varun Sharma, Avinash Narayanan, P. J. |
| author_facet | Manu, Pranav Srivastava, Astitva Raj, Amit Jampani, Varun Sharma, Avinash Narayanan, P. J. |
| contents | Creating photorealistic, animatable, and relightable 3D head avatars traditionally requires expensive Lightstage with multiple calibrated cameras, making it inaccessible for widespread adoption. To bridge this gap, we present a novel, cost-effective approach for creating high-quality relightable head avatars using only a smartphone equipped with polaroid filters. Our approach involves simultaneously capturing cross-polarized and parallel-polarized video streams in a dark room with a single point-light source, separating the skin's diffuse and specular components during dynamic facial performances. We introduce a hybrid representation that embeds 2D Gaussians in the UV space of a parametric head model, facilitating efficient real-time rendering while preserving high-fidelity geometric details. Our learning-based neural analysis-by-synthesis pipeline decouples pose and expression-dependent geometrical offsets from appearance, decomposing the surface into albedo, normal, and specular UV texture maps, along with the environment maps. We collect a unique dataset of various subjects performing diverse facial expressions and head movements. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_09671 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LightHeadEd: Relightable & Editable Head Avatars from a Smartphone Manu, Pranav Srivastava, Astitva Raj, Amit Jampani, Varun Sharma, Avinash Narayanan, P. J. Computer Vision and Pattern Recognition Creating photorealistic, animatable, and relightable 3D head avatars traditionally requires expensive Lightstage with multiple calibrated cameras, making it inaccessible for widespread adoption. To bridge this gap, we present a novel, cost-effective approach for creating high-quality relightable head avatars using only a smartphone equipped with polaroid filters. Our approach involves simultaneously capturing cross-polarized and parallel-polarized video streams in a dark room with a single point-light source, separating the skin's diffuse and specular components during dynamic facial performances. We introduce a hybrid representation that embeds 2D Gaussians in the UV space of a parametric head model, facilitating efficient real-time rendering while preserving high-fidelity geometric details. Our learning-based neural analysis-by-synthesis pipeline decouples pose and expression-dependent geometrical offsets from appearance, decomposing the surface into albedo, normal, and specular UV texture maps, along with the environment maps. We collect a unique dataset of various subjects performing diverse facial expressions and head movements. |
| title | LightHeadEd: Relightable & Editable Head Avatars from a Smartphone |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.09671 |