Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2508.17416 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912551940390912 |
|---|---|
| author | Ramos, Patrick Ramos, Ryan Garcia, Noa |
| author_facet | Ramos, Patrick Ramos, Ryan Garcia, Noa |
| contents | We analyze data leakage in visual datasets. Data leakage refers to images in evaluation benchmarks that have been seen during training, compromising fair model evaluation. Given that large-scale datasets are often sourced from the internet, where many computer vision benchmarks are publicly available, our efforts are focused into identifying and studying this phenomenon. We characterize visual leakage into different types according to its modality, coverage, and degree. By applying image retrieval techniques, we unequivocally show that all the analyzed datasets present some form of leakage, and that all types of leakage, from severe instances to more subtle cases, compromise the reliability of model evaluation in downstream tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_17416 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Data Leakage in Visual Datasets Ramos, Patrick Ramos, Ryan Garcia, Noa Computer Vision and Pattern Recognition We analyze data leakage in visual datasets. Data leakage refers to images in evaluation benchmarks that have been seen during training, compromising fair model evaluation. Given that large-scale datasets are often sourced from the internet, where many computer vision benchmarks are publicly available, our efforts are focused into identifying and studying this phenomenon. We characterize visual leakage into different types according to its modality, coverage, and degree. By applying image retrieval techniques, we unequivocally show that all the analyzed datasets present some form of leakage, and that all types of leakage, from severe instances to more subtle cases, compromise the reliability of model evaluation in downstream tasks. |
| title | Data Leakage in Visual Datasets |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2508.17416 |