Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Artículo Open Access |
| Published: |
Wiley
2025
|
| Subjects: | |
| Online Access: | https://onlinelibrary.wiley.com/doi/10.1002/sam.70021 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1867012421518884865 |
|---|---|
| author | Ngoc‐Ty Nguyen Larry Tang Lulu Chen P. Jonathon Phillips |
| author_facet | Ngoc‐Ty Nguyen Larry Tang Lulu Chen P. Jonathon Phillips Ngoc‐Ty Nguyen Larry Tang Lulu Chen P. Jonathon Phillips |
| collection | Wiley Open Access |
| contents | A Homogeneity Test for Ordinal Receiver Operating Characteristic Regression With Application to Facial Recognition Accuracy Assessment Ngoc‐Ty Nguyen Larry Tang Lulu Chen P. Jonathon Phillips Statistical Analysis and Data Mining: An ASA Data Science Journal ABSTRACT Ordinal scores occur commonly in medical imaging studies and more recently in black‐box studies on forensic identification accuracy. To assess the accuracy of radiologists in medical imaging studies or the accuracy of forensic examiners in biometric studies, one needs to estimate the accuracy measures such as the receiver operating characteristic (ROC) curves and also account for the covariates related to the radiologists or forensic examiners. The novelty of the paper is twofold. First, we propose a new covariate‐adjusted homogeneity test for ordinal ROC curves to determine differences in accuracy among multiple rater groups. Second, since the covariance structure among the ROC regression estimators is not available, we obtained the asymptotic covariance matrix of the ROC estimators and derived theoretical results of the proposed test. We conducted extensive simulation studies to evaluate the finite sample performance of the proposed test. The simulation results show that estimated ROC curves are consistent and the empirical coverage of the confidence intervals is close to the nominal level. Our proposed test is applied to a large‐scale face recognition study in which participants include facial examiners, facial reviewers, super‐recognizers, fingerprint examiners, and students. The results show differences in accuracy among five rater groups. Ad‐hoc pairwise comparison tests are then conducted by establishing confidence bands of differences among ROC curves. Those pairwise tests identify statistically significant differences in ROC curves among five participant groups. 10.1002/sam.70021 http://onlinelibrary.wiley.com/termsAndConditions#vor |
| doi_str_mv | 10.1002/sam.70021 |
| format | Artículo Open Access |
| id | wiley_oa_10_1002_sam_70021 |
| institution | Wiley Open Access |
| license_str_mv | http://onlinelibrary.wiley.com/termsAndConditions#vor |
| publishDate | 2025 |
| publisher | Wiley |
| record_format | wiley_oa |
| spellingShingle | A Homogeneity Test for Ordinal Receiver Operating Characteristic Regression With Application to Facial Recognition Accuracy Assessment Ngoc‐Ty Nguyen Larry Tang Lulu Chen P. Jonathon Phillips Statistical Analysis and Data Mining: An ASA Data Science Journal A Homogeneity Test for Ordinal Receiver Operating Characteristic Regression With Application to Facial Recognition Accuracy Assessment Ngoc‐Ty Nguyen Larry Tang Lulu Chen P. Jonathon Phillips Statistical Analysis and Data Mining: An ASA Data Science Journal ABSTRACT Ordinal scores occur commonly in medical imaging studies and more recently in black‐box studies on forensic identification accuracy. To assess the accuracy of radiologists in medical imaging studies or the accuracy of forensic examiners in biometric studies, one needs to estimate the accuracy measures such as the receiver operating characteristic (ROC) curves and also account for the covariates related to the radiologists or forensic examiners. The novelty of the paper is twofold. First, we propose a new covariate‐adjusted homogeneity test for ordinal ROC curves to determine differences in accuracy among multiple rater groups. Second, since the covariance structure among the ROC regression estimators is not available, we obtained the asymptotic covariance matrix of the ROC estimators and derived theoretical results of the proposed test. We conducted extensive simulation studies to evaluate the finite sample performance of the proposed test. The simulation results show that estimated ROC curves are consistent and the empirical coverage of the confidence intervals is close to the nominal level. Our proposed test is applied to a large‐scale face recognition study in which participants include facial examiners, facial reviewers, super‐recognizers, fingerprint examiners, and students. The results show differences in accuracy among five rater groups. Ad‐hoc pairwise comparison tests are then conducted by establishing confidence bands of differences among ROC curves. Those pairwise tests identify statistically significant differences in ROC curves among five participant groups. 10.1002/sam.70021 http://onlinelibrary.wiley.com/termsAndConditions#vor |
| title | A Homogeneity Test for Ordinal Receiver Operating Characteristic Regression With Application to Facial Recognition Accuracy Assessment |
| topic | Statistical Analysis and Data Mining: An ASA Data Science Journal |
| url | https://onlinelibrary.wiley.com/doi/10.1002/sam.70021 |