Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2507.01835 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908847988277248 |
|---|---|
| author | Reutsky, Daniil Vladimirov, Daniil Mamedov, Yasin Perevozchikov, Georgy Mehta, Nancy Ershov, Egor Timofte, Radu |
| author_facet | Reutsky, Daniil Vladimirov, Daniil Mamedov, Yasin Perevozchikov, Georgy Mehta, Nancy Ershov, Egor Timofte, Radu |
| contents | Hyperspectral reconstruction (HSR) from RGB images is a highly promising direction for accurate color reproduction and material color measurement. While most existing approaches rely on a single RGB image - thereby limiting reconstruction accuracy - the majority of modern smartphones are equipped with two or more cameras. In this work, we propose a novel multi-image-to-hyperspectral reconstruction (MI-HSR) framework that leverages a triple-camera smartphone system, where two lenses are equipped with carefully selected spectral filters. Our easy-to-implement configuration, based on theoretical and empirical analysis, allows to obtain more complete and diverse spectral data than traditional single-chamber setups. To support this new paradigm, we introduce Doomer, the first dataset for MI-HSR, comprising aligned images from three smartphone cameras and a hyperspectral reference camera across diverse scenes. We further introduce a lightweight alignment module for MI-HSR that effectively fuses multi-view inputs while mitigating parallax- and occlusion-induced artifacts. Proposed module demonstrate consistent quality improvements for modern HSR methods. In a nutshell, our setup allows 30% more accurate estimations of spectra compared to an ordinary RGB camera, while the proposed alignment module boosts the reconstruction quality of SotA methods by an additional 5%. Our findings suggest that spectral filtering of multiple views with commodity hardware unlocks more accurate and practical hyperspectral imaging. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_01835 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Modulate and Reconstruct: Learning Hyperspectral Imaging from Misaligned Smartphone Views Reutsky, Daniil Vladimirov, Daniil Mamedov, Yasin Perevozchikov, Georgy Mehta, Nancy Ershov, Egor Timofte, Radu Computer Vision and Pattern Recognition Hyperspectral reconstruction (HSR) from RGB images is a highly promising direction for accurate color reproduction and material color measurement. While most existing approaches rely on a single RGB image - thereby limiting reconstruction accuracy - the majority of modern smartphones are equipped with two or more cameras. In this work, we propose a novel multi-image-to-hyperspectral reconstruction (MI-HSR) framework that leverages a triple-camera smartphone system, where two lenses are equipped with carefully selected spectral filters. Our easy-to-implement configuration, based on theoretical and empirical analysis, allows to obtain more complete and diverse spectral data than traditional single-chamber setups. To support this new paradigm, we introduce Doomer, the first dataset for MI-HSR, comprising aligned images from three smartphone cameras and a hyperspectral reference camera across diverse scenes. We further introduce a lightweight alignment module for MI-HSR that effectively fuses multi-view inputs while mitigating parallax- and occlusion-induced artifacts. Proposed module demonstrate consistent quality improvements for modern HSR methods. In a nutshell, our setup allows 30% more accurate estimations of spectra compared to an ordinary RGB camera, while the proposed alignment module boosts the reconstruction quality of SotA methods by an additional 5%. Our findings suggest that spectral filtering of multiple views with commodity hardware unlocks more accurate and practical hyperspectral imaging. |
| title | Modulate and Reconstruct: Learning Hyperspectral Imaging from Misaligned Smartphone Views |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2507.01835 |