One Patch is All You Need: Joint Surface Material Reconstruction and Classification from Minimal Visual Cues
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911287261265920 |
|---|---|
| author | Penchala, Sindhuja Money, Gavin Marques, Gabriel Wood, Samuel Kirschman, Jessica Atkison, Travis Rahimi, Shahram Golilarz, Noorbakhsh Amiri |
| author_facet | Penchala, Sindhuja Money, Gavin Marques, Gabriel Wood, Samuel Kirschman, Jessica Atkison, Travis Rahimi, Shahram Golilarz, Noorbakhsh Amiri |
| contents | Understanding material surfaces from sparse visual cues is critical for applications in robotics, simulation, and material perception. However, most existing methods rely on dense or full-scene observations, limiting their effectiveness in constrained or partial view environment. To address this challenge, we introduce SMARC, a unified model for Surface MAterial Reconstruction and Classification from minimal visual input. By giving only a single 10% contiguous patch of the image, SMARC recognizes and reconstructs the full RGB surface while simultaneously classifying the material category. Our architecture combines a Partial Convolutional U-Net with a classification head, enabling both spatial inpainting and semantic understanding under extreme observation sparsity. We compared SMARC against five models including convolutional autoencoders [17], Vision Transformer (ViT) [13], Masked Autoencoder (MAE) [5], Swin Transformer [9], and DETR [2] using Touch and Go dataset [16] of real-world surface textures. SMARC achieves state-of-the-art results with a PSNR of 17.55 dB and a material classification accuracy of 85.10%. Our findings highlight the advantages of partial convolution in spatial reasoning under missing data and establish a strong foundation for minimal-vision surface understanding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_20784 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | One Patch is All You Need: Joint Surface Material Reconstruction and Classification from Minimal Visual Cues Penchala, Sindhuja Money, Gavin Marques, Gabriel Wood, Samuel Kirschman, Jessica Atkison, Travis Rahimi, Shahram Golilarz, Noorbakhsh Amiri Computer Vision and Pattern Recognition Understanding material surfaces from sparse visual cues is critical for applications in robotics, simulation, and material perception. However, most existing methods rely on dense or full-scene observations, limiting their effectiveness in constrained or partial view environment. To address this challenge, we introduce SMARC, a unified model for Surface MAterial Reconstruction and Classification from minimal visual input. By giving only a single 10% contiguous patch of the image, SMARC recognizes and reconstructs the full RGB surface while simultaneously classifying the material category. Our architecture combines a Partial Convolutional U-Net with a classification head, enabling both spatial inpainting and semantic understanding under extreme observation sparsity. We compared SMARC against five models including convolutional autoencoders [17], Vision Transformer (ViT) [13], Masked Autoencoder (MAE) [5], Swin Transformer [9], and DETR [2] using Touch and Go dataset [16] of real-world surface textures. SMARC achieves state-of-the-art results with a PSNR of 17.55 dB and a material classification accuracy of 85.10%. Our findings highlight the advantages of partial convolution in spatial reasoning under missing data and establish a strong foundation for minimal-vision surface understanding. |
| title | One Patch is All You Need: Joint Surface Material Reconstruction and Classification from Minimal Visual Cues |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2511.20784 |