GHNeRF: Learning Generalizable Human Features with Efficient Neural Radiance Fields
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914746144391168 |
|---|---|
| author | Dey, Arnab Yang, Di Agaram, Rohith Dantcheva, Antitza Comport, Andrew I. Sridhar, Srinath Martinet, Jean |
| author_facet | Dey, Arnab Yang, Di Agaram, Rohith Dantcheva, Antitza Comport, Andrew I. Sridhar, Srinath Martinet, Jean |
| contents | Recent advances in Neural Radiance Fields (NeRF) have demonstrated promising results in 3D scene representations, including 3D human representations. However, these representations often lack crucial information on the underlying human pose and structure, which is crucial for AR/VR applications and games. In this paper, we introduce a novel approach, termed GHNeRF, designed to address these limitations by learning 2D/3D joint locations of human subjects with NeRF representation. GHNeRF uses a pre-trained 2D encoder streamlined to extract essential human features from 2D images, which are then incorporated into the NeRF framework in order to encode human biomechanic features. This allows our network to simultaneously learn biomechanic features, such as joint locations, along with human geometry and texture. To assess the effectiveness of our method, we conduct a comprehensive comparison with state-of-the-art human NeRF techniques and joint estimation algorithms. Our results show that GHNeRF can achieve state-of-the-art results in near real-time. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_06246 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | GHNeRF: Learning Generalizable Human Features with Efficient Neural Radiance Fields Dey, Arnab Yang, Di Agaram, Rohith Dantcheva, Antitza Comport, Andrew I. Sridhar, Srinath Martinet, Jean Computer Vision and Pattern Recognition Artificial Intelligence Recent advances in Neural Radiance Fields (NeRF) have demonstrated promising results in 3D scene representations, including 3D human representations. However, these representations often lack crucial information on the underlying human pose and structure, which is crucial for AR/VR applications and games. In this paper, we introduce a novel approach, termed GHNeRF, designed to address these limitations by learning 2D/3D joint locations of human subjects with NeRF representation. GHNeRF uses a pre-trained 2D encoder streamlined to extract essential human features from 2D images, which are then incorporated into the NeRF framework in order to encode human biomechanic features. This allows our network to simultaneously learn biomechanic features, such as joint locations, along with human geometry and texture. To assess the effectiveness of our method, we conduct a comprehensive comparison with state-of-the-art human NeRF techniques and joint estimation algorithms. Our results show that GHNeRF can achieve state-of-the-art results in near real-time. |
| title | GHNeRF: Learning Generalizable Human Features with Efficient Neural Radiance Fields |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2404.06246 |