UnMix-NeRF: Spectral Unmixing Meets Neural Radiance Fields
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911093525315584 |
|---|---|
| author | Perez, Fabian Rojas, Sara Hinojosa, Carlos Rueda-Chacón, Hoover Ghanem, Bernard |
| author_facet | Perez, Fabian Rojas, Sara Hinojosa, Carlos Rueda-Chacón, Hoover Ghanem, Bernard |
| contents | Neural Radiance Field (NeRF)-based segmentation methods focus on object semantics and rely solely on RGB data, lacking intrinsic material properties. This limitation restricts accurate material perception, which is crucial for robotics, augmented reality, simulation, and other applications. We introduce UnMix-NeRF, a framework that integrates spectral unmixing into NeRF, enabling joint hyperspectral novel view synthesis and unsupervised material segmentation. Our method models spectral reflectance via diffuse and specular components, where a learned dictionary of global endmembers represents pure material signatures, and per-point abundances capture their distribution. For material segmentation, we use spectral signature predictions along learned endmembers, allowing unsupervised material clustering. Additionally, UnMix-NeRF enables scene editing by modifying learned endmember dictionaries for flexible material-based appearance manipulation. Extensive experiments validate our approach, demonstrating superior spectral reconstruction and material segmentation to existing methods. Project page: https://www.factral.co/UnMix-NeRF. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_21884 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | UnMix-NeRF: Spectral Unmixing Meets Neural Radiance Fields Perez, Fabian Rojas, Sara Hinojosa, Carlos Rueda-Chacón, Hoover Ghanem, Bernard Image and Video Processing Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning Signal Processing Neural Radiance Field (NeRF)-based segmentation methods focus on object semantics and rely solely on RGB data, lacking intrinsic material properties. This limitation restricts accurate material perception, which is crucial for robotics, augmented reality, simulation, and other applications. We introduce UnMix-NeRF, a framework that integrates spectral unmixing into NeRF, enabling joint hyperspectral novel view synthesis and unsupervised material segmentation. Our method models spectral reflectance via diffuse and specular components, where a learned dictionary of global endmembers represents pure material signatures, and per-point abundances capture their distribution. For material segmentation, we use spectral signature predictions along learned endmembers, allowing unsupervised material clustering. Additionally, UnMix-NeRF enables scene editing by modifying learned endmember dictionaries for flexible material-based appearance manipulation. Extensive experiments validate our approach, demonstrating superior spectral reconstruction and material segmentation to existing methods. Project page: https://www.factral.co/UnMix-NeRF. |
| title | UnMix-NeRF: Spectral Unmixing Meets Neural Radiance Fields |
| topic | Image and Video Processing Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning Signal Processing |
| url | https://arxiv.org/abs/2506.21884 |