Self-Supervised Learning with a Multi-Task Latent Space Objective
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918324488634368 |
|---|---|
| author | De Plaen, Pierre-François Jha, Abhishek Van Gool, Luc Tuytelaars, Tinne Proesmans, Marc |
| author_facet | De Plaen, Pierre-François Jha, Abhishek Van Gool, Luc Tuytelaars, Tinne Proesmans, Marc |
| contents | Self-supervised learning (SSL) methods based on Siamese networks learn visual representations by aligning different views of the same image. The multi-crop strategy, which incorporates small local crops to global ones, enhances many SSL frameworks but causes instability in predictor-based architectures such as BYOL, SimSiam, and MoCo v3. We trace this failure to the shared predictor used across all views and demonstrate that assigning a separate predictor to each view type stabilizes multi-crop training, resulting in significant performance gains. Extending this idea, we treat each spatial transformation as a distinct alignment task and add cutout views, where part of the image is masked before encoding. This yields a simple multi-task formulation of asymmetric Siamese SSL that combines global, local, and masked views into a single framework. The approach is stable, generally applicable across backbones, and consistently improves the performance of ResNet and ViT models on ImageNet. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_05845 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Self-Supervised Learning with a Multi-Task Latent Space Objective De Plaen, Pierre-François Jha, Abhishek Van Gool, Luc Tuytelaars, Tinne Proesmans, Marc Computer Vision and Pattern Recognition Self-supervised learning (SSL) methods based on Siamese networks learn visual representations by aligning different views of the same image. The multi-crop strategy, which incorporates small local crops to global ones, enhances many SSL frameworks but causes instability in predictor-based architectures such as BYOL, SimSiam, and MoCo v3. We trace this failure to the shared predictor used across all views and demonstrate that assigning a separate predictor to each view type stabilizes multi-crop training, resulting in significant performance gains. Extending this idea, we treat each spatial transformation as a distinct alignment task and add cutout views, where part of the image is masked before encoding. This yields a simple multi-task formulation of asymmetric Siamese SSL that combines global, local, and masked views into a single framework. The approach is stable, generally applicable across backbones, and consistently improves the performance of ResNet and ViT models on ImageNet. |
| title | Self-Supervised Learning with a Multi-Task Latent Space Objective |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2602.05845 |