Using Knowledge Graphs to harvest datasets for efficient CLIP model training
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918151150632960 |
|---|---|
| author | Ging, Simon Walter, Sebastian Bratulić, Jelena Dienert, Johannes Bast, Hannah Brox, Thomas |
| author_facet | Ging, Simon Walter, Sebastian Bratulić, Jelena Dienert, Johannes Bast, Hannah Brox, Thomas |
| contents | Training high-quality CLIP models typically requires enormous datasets, which limits the development of domain-specific models -- especially in areas that even the largest CLIP models do not cover well -- and drives up training costs. This poses challenges for scientific research that needs fine-grained control over the training procedure of CLIP models. In this work, we show that by employing smart web search strategies enhanced with knowledge graphs, a robust CLIP model can be trained from scratch with considerably less data. Specifically, we demonstrate that an expert foundation model for living organisms can be built using just 10M images. Moreover, we introduce EntityNet, a dataset comprising 33M images paired with 46M text descriptions, which enables the training of a generic CLIP model in significantly reduced time. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_02746 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Using Knowledge Graphs to harvest datasets for efficient CLIP model training Ging, Simon Walter, Sebastian Bratulić, Jelena Dienert, Johannes Bast, Hannah Brox, Thomas Computer Vision and Pattern Recognition Computation and Language Information Retrieval Machine Learning Training high-quality CLIP models typically requires enormous datasets, which limits the development of domain-specific models -- especially in areas that even the largest CLIP models do not cover well -- and drives up training costs. This poses challenges for scientific research that needs fine-grained control over the training procedure of CLIP models. In this work, we show that by employing smart web search strategies enhanced with knowledge graphs, a robust CLIP model can be trained from scratch with considerably less data. Specifically, we demonstrate that an expert foundation model for living organisms can be built using just 10M images. Moreover, we introduce EntityNet, a dataset comprising 33M images paired with 46M text descriptions, which enables the training of a generic CLIP model in significantly reduced time. |
| title | Using Knowledge Graphs to harvest datasets for efficient CLIP model training |
| topic | Computer Vision and Pattern Recognition Computation and Language Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2505.02746 |