Self-Supervised Vision Transformers Are Efficient Segmentation Learners for Imperfect Labels
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914649112313856 |
|---|---|
| author | Lee, Seungho Kang, Seoungyoon Shim, Hyunjung |
| author_facet | Lee, Seungho Kang, Seoungyoon Shim, Hyunjung |
| contents | This study demonstrates a cost-effective approach to semantic segmentation using self-supervised vision transformers (SSVT). By freezing the SSVT backbone and training a lightweight segmentation head, our approach effectively utilizes imperfect labels, thereby improving robustness to label imperfections. Empirical experiments show significant performance improvements over existing methods for various annotation types, including scribble, point-level, and image-level labels. The research highlights the effectiveness of self-supervised vision transformers in dealing with imperfect labels, providing a practical and efficient solution for semantic segmentation while reducing annotation costs. Through extensive experiments, we confirm that our method outperforms baseline models for all types of imperfect labels. Especially under the zero-shot vision-language-model-based label, our model exhibits 11.5\%p performance gain compared to the baseline. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_12535 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Self-Supervised Vision Transformers Are Efficient Segmentation Learners for Imperfect Labels Lee, Seungho Kang, Seoungyoon Shim, Hyunjung Computer Vision and Pattern Recognition This study demonstrates a cost-effective approach to semantic segmentation using self-supervised vision transformers (SSVT). By freezing the SSVT backbone and training a lightweight segmentation head, our approach effectively utilizes imperfect labels, thereby improving robustness to label imperfections. Empirical experiments show significant performance improvements over existing methods for various annotation types, including scribble, point-level, and image-level labels. The research highlights the effectiveness of self-supervised vision transformers in dealing with imperfect labels, providing a practical and efficient solution for semantic segmentation while reducing annotation costs. Through extensive experiments, we confirm that our method outperforms baseline models for all types of imperfect labels. Especially under the zero-shot vision-language-model-based label, our model exhibits 11.5\%p performance gain compared to the baseline. |
| title | Self-Supervised Vision Transformers Are Efficient Segmentation Learners for Imperfect Labels |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2401.12535 |