Reconstructing Training Data From Real World Models Trained with Transfer Learning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929430752919552 |
|---|---|
| author | Oz, Yakir Yehudai, Gilad Vardi, Gal Antebi, Itai Irani, Michal Haim, Niv |
| author_facet | Oz, Yakir Yehudai, Gilad Vardi, Gal Antebi, Itai Irani, Michal Haim, Niv |
| contents | Current methods for reconstructing training data from trained classifiers are restricted to very small models, limited training set sizes, and low-resolution images. Such restrictions hinder their applicability to real-world scenarios. In this paper, we present a novel approach enabling data reconstruction in realistic settings for models trained on high-resolution images. Our method adapts the reconstruction scheme of arXiv:2206.07758 to real-world scenarios -- specifically, targeting models trained via transfer learning over image embeddings of large pre-trained models like DINO-ViT and CLIP. Our work employs data reconstruction in the embedding space rather than in the image space, showcasing its applicability beyond visual data. Moreover, we introduce a novel clustering-based method to identify good reconstructions from thousands of candidates. This significantly improves on previous works that relied on knowledge of the training set to identify good reconstructed images. Our findings shed light on a potential privacy risk for data leakage from models trained using transfer learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_15845 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Reconstructing Training Data From Real World Models Trained with Transfer Learning Oz, Yakir Yehudai, Gilad Vardi, Gal Antebi, Itai Irani, Michal Haim, Niv Machine Learning Artificial Intelligence Cryptography and Security Computer Vision and Pattern Recognition Current methods for reconstructing training data from trained classifiers are restricted to very small models, limited training set sizes, and low-resolution images. Such restrictions hinder their applicability to real-world scenarios. In this paper, we present a novel approach enabling data reconstruction in realistic settings for models trained on high-resolution images. Our method adapts the reconstruction scheme of arXiv:2206.07758 to real-world scenarios -- specifically, targeting models trained via transfer learning over image embeddings of large pre-trained models like DINO-ViT and CLIP. Our work employs data reconstruction in the embedding space rather than in the image space, showcasing its applicability beyond visual data. Moreover, we introduce a novel clustering-based method to identify good reconstructions from thousands of candidates. This significantly improves on previous works that relied on knowledge of the training set to identify good reconstructed images. Our findings shed light on a potential privacy risk for data leakage from models trained using transfer learning. |
| title | Reconstructing Training Data From Real World Models Trained with Transfer Learning |
| topic | Machine Learning Artificial Intelligence Cryptography and Security Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.15845 |