Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.17035 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909654304423936 |
|---|---|
| author | Matos, João Van Calster, Ben Celi, Leo Anthony Dhiman, Paula Gichoya, Judy Wawira Riley, Richard D. Russell, Chris Khalid, Sara Collins, Gary S. |
| author_facet | Matos, João Van Calster, Ben Celi, Leo Anthony Dhiman, Paula Gichoya, Judy Wawira Riley, Richard D. Russell, Chris Khalid, Sara Collins, Gary S. |
| contents | Predictive artificial intelligence (AI) offers an opportunity to improve clinical practice and patient outcomes, but risks perpetuating biases if fairness is inadequately addressed. However, the definition of "fairness" remains unclear. We conducted a scoping review to identify and critically appraise fairness metrics for clinical predictive AI. We defined a "fairness metric" as a measure quantifying whether a model discriminates (societally) against individuals or groups defined by sensitive attributes. We searched five databases (2014-2024), screening 820 records, to include 41 studies, and extracted 62 fairness metrics. Metrics were classified by performance-dependency, model output level, and base performance metric, revealing a fragmented landscape with limited clinical validation and overreliance on threshold-dependent measures. Eighteen metrics were explicitly developed for healthcare, including only one clinical utility metric. Our findings highlight conceptual challenges in defining and quantifying fairness and identify gaps in uncertainty quantification, intersectionality, and real-world applicability. Future work should prioritise clinically meaningful metrics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_17035 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Critical Appraisal of Fairness Metrics in Clinical Predictive AI Matos, João Van Calster, Ben Celi, Leo Anthony Dhiman, Paula Gichoya, Judy Wawira Riley, Richard D. Russell, Chris Khalid, Sara Collins, Gary S. Machine Learning Predictive artificial intelligence (AI) offers an opportunity to improve clinical practice and patient outcomes, but risks perpetuating biases if fairness is inadequately addressed. However, the definition of "fairness" remains unclear. We conducted a scoping review to identify and critically appraise fairness metrics for clinical predictive AI. We defined a "fairness metric" as a measure quantifying whether a model discriminates (societally) against individuals or groups defined by sensitive attributes. We searched five databases (2014-2024), screening 820 records, to include 41 studies, and extracted 62 fairness metrics. Metrics were classified by performance-dependency, model output level, and base performance metric, revealing a fragmented landscape with limited clinical validation and overreliance on threshold-dependent measures. Eighteen metrics were explicitly developed for healthcare, including only one clinical utility metric. Our findings highlight conceptual challenges in defining and quantifying fairness and identify gaps in uncertainty quantification, intersectionality, and real-world applicability. Future work should prioritise clinically meaningful metrics. |
| title | Critical Appraisal of Fairness Metrics in Clinical Predictive AI |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2506.17035 |