Semi-Supervised Fine-Tuning of Vision Foundation Models with Content-Style Decomposition
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912058216284160 |
|---|---|
| author | Drozdova, Mariia Kinakh, Vitaliy Belousov, Yury Lastufka, Erica Voloshynovskiy, Slava |
| author_facet | Drozdova, Mariia Kinakh, Vitaliy Belousov, Yury Lastufka, Erica Voloshynovskiy, Slava |
| contents | In this paper, we present a semi-supervised fine-tuning approach designed to improve the performance of pre-trained foundation models on downstream tasks with limited labeled data. By leveraging content-style decomposition within an information-theoretic framework, our method enhances the latent representations of pre-trained vision foundation models, aligning them more effectively with specific task objectives and addressing the problem of distribution shift. We evaluate our approach on multiple datasets, including MNIST, its augmented variations (with yellow and white stripes), CIFAR-10, SVHN, and GalaxyMNIST. The experiments show improvements over supervised finetuning baseline of pre-trained models, particularly in low-labeled data regimes, across both frozen and trainable backbones for the majority of the tested datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_02069 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Semi-Supervised Fine-Tuning of Vision Foundation Models with Content-Style Decomposition Drozdova, Mariia Kinakh, Vitaliy Belousov, Yury Lastufka, Erica Voloshynovskiy, Slava Computer Vision and Pattern Recognition Machine Learning In this paper, we present a semi-supervised fine-tuning approach designed to improve the performance of pre-trained foundation models on downstream tasks with limited labeled data. By leveraging content-style decomposition within an information-theoretic framework, our method enhances the latent representations of pre-trained vision foundation models, aligning them more effectively with specific task objectives and addressing the problem of distribution shift. We evaluate our approach on multiple datasets, including MNIST, its augmented variations (with yellow and white stripes), CIFAR-10, SVHN, and GalaxyMNIST. The experiments show improvements over supervised finetuning baseline of pre-trained models, particularly in low-labeled data regimes, across both frozen and trainable backbones for the majority of the tested datasets. |
| title | Semi-Supervised Fine-Tuning of Vision Foundation Models with Content-Style Decomposition |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2410.02069 |