Joint-Embedding Predictive Architecture for Self-Supervised Learning of Mask Classification Architecture
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917722261028864 |
|---|---|
| author | Kim, Dong-Hee Cho, Sungduk Cho, Hyeonwoo Park, Chanmin Kim, Jinyoung Kim, Won Hwa |
| author_facet | Kim, Dong-Hee Cho, Sungduk Cho, Hyeonwoo Park, Chanmin Kim, Jinyoung Kim, Won Hwa |
| contents | In this work, we introduce Mask-JEPA, a self-supervised learning framework tailored for mask classification architectures (MCA), to overcome the traditional constraints associated with training segmentation models. Mask-JEPA combines a Joint Embedding Predictive Architecture with MCA to adeptly capture intricate semantics and precise object boundaries. Our approach addresses two critical challenges in self-supervised learning: 1) extracting comprehensive representations for universal image segmentation from a pixel decoder, and 2) effectively training the transformer decoder. The use of the transformer decoder as a predictor within the JEPA framework allows proficient training in universal image segmentation tasks. Through rigorous evaluations on datasets such as ADE20K, Cityscapes and COCO, Mask-JEPA demonstrates not only competitive results but also exceptional adaptability and robustness across various training scenarios. The architecture-agnostic nature of Mask-JEPA further underscores its versatility, allowing seamless adaptation to various mask classification family. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_10733 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Joint-Embedding Predictive Architecture for Self-Supervised Learning of Mask Classification Architecture Kim, Dong-Hee Cho, Sungduk Cho, Hyeonwoo Park, Chanmin Kim, Jinyoung Kim, Won Hwa Computer Vision and Pattern Recognition In this work, we introduce Mask-JEPA, a self-supervised learning framework tailored for mask classification architectures (MCA), to overcome the traditional constraints associated with training segmentation models. Mask-JEPA combines a Joint Embedding Predictive Architecture with MCA to adeptly capture intricate semantics and precise object boundaries. Our approach addresses two critical challenges in self-supervised learning: 1) extracting comprehensive representations for universal image segmentation from a pixel decoder, and 2) effectively training the transformer decoder. The use of the transformer decoder as a predictor within the JEPA framework allows proficient training in universal image segmentation tasks. Through rigorous evaluations on datasets such as ADE20K, Cityscapes and COCO, Mask-JEPA demonstrates not only competitive results but also exceptional adaptability and robustness across various training scenarios. The architecture-agnostic nature of Mask-JEPA further underscores its versatility, allowing seamless adaptation to various mask classification family. |
| title | Joint-Embedding Predictive Architecture for Self-Supervised Learning of Mask Classification Architecture |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.10733 |