Intrinsic Self-Supervision for Data Quality Audits
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916457147793408 |
|---|---|
| author | Gröger, Fabian Lionetti, Simone Gottfrois, Philippe Gonzalez-Jimenez, Alvaro Amruthalingam, Ludovic Consortium, Labelling Groh, Matthew Navarini, Alexander A. Pouly, Marc |
| author_facet | Gröger, Fabian Lionetti, Simone Gottfrois, Philippe Gonzalez-Jimenez, Alvaro Amruthalingam, Ludovic Consortium, Labelling Groh, Matthew Navarini, Alexander A. Pouly, Marc |
| contents | Benchmark datasets in computer vision often contain off-topic images, near duplicates, and label errors, leading to inaccurate estimates of model performance. In this paper, we revisit the task of data cleaning and formalize it as either a ranking problem, which significantly reduces human inspection effort, or a scoring problem, which allows for automated decisions based on score distributions. We find that a specific combination of context-aware self-supervised representation learning and distance-based indicators is effective in finding issues without annotation biases. This methodology, which we call SelfClean, surpasses state-of-the-art performance in detecting off-topic images, near duplicates, and label errors within widely-used image datasets, such as ImageNet-1k, Food-101N, and STL-10, both for synthetic issues and real contamination. We apply the detailed method to multiple image benchmarks, identify up to 16% of issues, and confirm an improvement in evaluation reliability upon cleaning. The official implementation can be found at: https://github.com/Digital-Dermatology/SelfClean. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_17048 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Intrinsic Self-Supervision for Data Quality Audits Gröger, Fabian Lionetti, Simone Gottfrois, Philippe Gonzalez-Jimenez, Alvaro Amruthalingam, Ludovic Consortium, Labelling Groh, Matthew Navarini, Alexander A. Pouly, Marc Computer Vision and Pattern Recognition Benchmark datasets in computer vision often contain off-topic images, near duplicates, and label errors, leading to inaccurate estimates of model performance. In this paper, we revisit the task of data cleaning and formalize it as either a ranking problem, which significantly reduces human inspection effort, or a scoring problem, which allows for automated decisions based on score distributions. We find that a specific combination of context-aware self-supervised representation learning and distance-based indicators is effective in finding issues without annotation biases. This methodology, which we call SelfClean, surpasses state-of-the-art performance in detecting off-topic images, near duplicates, and label errors within widely-used image datasets, such as ImageNet-1k, Food-101N, and STL-10, both for synthetic issues and real contamination. We apply the detailed method to multiple image benchmarks, identify up to 16% of issues, and confirm an improvement in evaluation reliability upon cleaning. The official implementation can be found at: https://github.com/Digital-Dermatology/SelfClean. |
| title | Intrinsic Self-Supervision for Data Quality Audits |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2305.17048 |