Goldilocks Test Sets for Face Verification
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914378820878336 |
|---|---|
| author | Wu, Haiyu Tian, Sicong Bhatta, Aman Gutierrez, Jacob Bezold, Grace Argueta, Genesis Ricanek Jr., Karl King, Michael C. Bowyer, Kevin W. |
| author_facet | Wu, Haiyu Tian, Sicong Bhatta, Aman Gutierrez, Jacob Bezold, Grace Argueta, Genesis Ricanek Jr., Karl King, Michael C. Bowyer, Kevin W. |
| contents | Reported face verification accuracy has reached a plateau on current well-known test sets. As a result, some difficult test sets have been assembled by reducing the image quality or adding artifacts to the image. However, we argue that test sets can be challenging without artificially reducing the image quality because the face recognition (FR) models suffer from correctly recognizing 1) the pairs from the same identity (i.e., genuine pairs) with a large face attribute difference, 2) the pairs from different identities (i.e., impostor pairs) with a small face attribute difference, and 3) the pairs of similar-looking identities (e.g., twins and relatives). We propose three challenging test sets to reveal important but ignored weaknesses of the existing FR algorithms. To challenge models on variation of facial attributes, we propose Hadrian and Eclipse to address facial hair differences and face exposure differences. The images in both test sets are high-quality and collected in a controlled environment. To challenge FR models on similar-looking persons, we propose ND-Twins, which contains images from a dedicated twins dataset. The LFW test protocol is used to structure the proposed test sets. Moreover, we introduce additional rules to assemble ``Goldilocks\footnote{https://en.wikipedia.org/wiki/Goldilocks_and_the_Three_Bears}" level test sets, including 1) restricted number of occurrence of hard samples, 2) equal chance evaluation across demographic groups, and 3) constrained identity overlap across validation folds. Quantitatively, without further processing the images, the proposed test sets have on-par or higher difficulties than the existing test sets that add artifacts to the images. The datasets are available at: https://github.com/HaiyuWu/SOTA-Face-Recognition-Train-and-Test. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_15965 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Goldilocks Test Sets for Face Verification Wu, Haiyu Tian, Sicong Bhatta, Aman Gutierrez, Jacob Bezold, Grace Argueta, Genesis Ricanek Jr., Karl King, Michael C. Bowyer, Kevin W. Computer Vision and Pattern Recognition Reported face verification accuracy has reached a plateau on current well-known test sets. As a result, some difficult test sets have been assembled by reducing the image quality or adding artifacts to the image. However, we argue that test sets can be challenging without artificially reducing the image quality because the face recognition (FR) models suffer from correctly recognizing 1) the pairs from the same identity (i.e., genuine pairs) with a large face attribute difference, 2) the pairs from different identities (i.e., impostor pairs) with a small face attribute difference, and 3) the pairs of similar-looking identities (e.g., twins and relatives). We propose three challenging test sets to reveal important but ignored weaknesses of the existing FR algorithms. To challenge models on variation of facial attributes, we propose Hadrian and Eclipse to address facial hair differences and face exposure differences. The images in both test sets are high-quality and collected in a controlled environment. To challenge FR models on similar-looking persons, we propose ND-Twins, which contains images from a dedicated twins dataset. The LFW test protocol is used to structure the proposed test sets. Moreover, we introduce additional rules to assemble ``Goldilocks\footnote{https://en.wikipedia.org/wiki/Goldilocks_and_the_Three_Bears}" level test sets, including 1) restricted number of occurrence of hard samples, 2) equal chance evaluation across demographic groups, and 3) constrained identity overlap across validation folds. Quantitatively, without further processing the images, the proposed test sets have on-par or higher difficulties than the existing test sets that add artifacts to the images. The datasets are available at: https://github.com/HaiyuWu/SOTA-Face-Recognition-Train-and-Test. |
| title | Goldilocks Test Sets for Face Verification |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.15965 |