Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2410.19836 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916883553320960 |
|---|---|
| author | Docherty, Ronan Vamvakeros, Antonis Cooper, Samuel J. |
| author_facet | Docherty, Ronan Vamvakeros, Antonis Cooper, Samuel J. |
| contents | The features of self-supervised vision transformers (ViTs) contain strong semantic and positional information relevant to downstream tasks like object localization and segmentation. Recent works combine these features with traditional methods like clustering, graph partitioning or region correlations to achieve impressive baselines without finetuning or training additional networks. We leverage upsampled features from ViT networks (e.g DINOv2) in two workflows: in a clustering based approach for object localization and segmentation, and paired with standard classifiers in weakly supervised materials segmentation. Both show strong performance on benchmarks, especially in weakly supervised segmentation where the ViT features capture complex relationships inaccessible to classical approaches. We expect the flexibility and generalizability of these features will both speed up and strengthen materials characterization, from segmentation to property-prediction. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_19836 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Upsampling DINOv2 features for unsupervised vision tasks and weakly supervised materials segmentation Docherty, Ronan Vamvakeros, Antonis Cooper, Samuel J. Computer Vision and Pattern Recognition Materials Science Image and Video Processing The features of self-supervised vision transformers (ViTs) contain strong semantic and positional information relevant to downstream tasks like object localization and segmentation. Recent works combine these features with traditional methods like clustering, graph partitioning or region correlations to achieve impressive baselines without finetuning or training additional networks. We leverage upsampled features from ViT networks (e.g DINOv2) in two workflows: in a clustering based approach for object localization and segmentation, and paired with standard classifiers in weakly supervised materials segmentation. Both show strong performance on benchmarks, especially in weakly supervised segmentation where the ViT features capture complex relationships inaccessible to classical approaches. We expect the flexibility and generalizability of these features will both speed up and strengthen materials characterization, from segmentation to property-prediction. |
| title | Upsampling DINOv2 features for unsupervised vision tasks and weakly supervised materials segmentation |
| topic | Computer Vision and Pattern Recognition Materials Science Image and Video Processing |
| url | https://arxiv.org/abs/2410.19836 |