Perils of Label Indeterminacy: A Case Study on Prediction of Neurological Recovery After Cardiac Arrest
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915277103431680 |
|---|---|
| author | Schoeffer, Jakob De-Arteaga, Maria Elmer, Jonathan |
| author_facet | Schoeffer, Jakob De-Arteaga, Maria Elmer, Jonathan |
| contents | The design of AI systems to assist human decision-making typically requires the availability of labels to train and evaluate supervised models. Frequently, however, these labels are unknown, and different ways of estimating them involve unverifiable assumptions or arbitrary choices. In this work, we introduce the concept of label indeterminacy and derive important implications in high-stakes AI-assisted decision-making. We present an empirical study in a healthcare context, focusing specifically on predicting the recovery of comatose patients after resuscitation from cardiac arrest. Our study shows that label indeterminacy can result in models that perform similarly when evaluated on patients with known labels, but vary drastically in their predictions for patients where labels are unknown. After demonstrating crucial ethical implications of label indeterminacy in this high-stakes context, we discuss takeaways for evaluation, reporting, and design. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_04243 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Perils of Label Indeterminacy: A Case Study on Prediction of Neurological Recovery After Cardiac Arrest Schoeffer, Jakob De-Arteaga, Maria Elmer, Jonathan Machine Learning Artificial Intelligence Human-Computer Interaction Methodology The design of AI systems to assist human decision-making typically requires the availability of labels to train and evaluate supervised models. Frequently, however, these labels are unknown, and different ways of estimating them involve unverifiable assumptions or arbitrary choices. In this work, we introduce the concept of label indeterminacy and derive important implications in high-stakes AI-assisted decision-making. We present an empirical study in a healthcare context, focusing specifically on predicting the recovery of comatose patients after resuscitation from cardiac arrest. Our study shows that label indeterminacy can result in models that perform similarly when evaluated on patients with known labels, but vary drastically in their predictions for patients where labels are unknown. After demonstrating crucial ethical implications of label indeterminacy in this high-stakes context, we discuss takeaways for evaluation, reporting, and design. |
| title | Perils of Label Indeterminacy: A Case Study on Prediction of Neurological Recovery After Cardiac Arrest |
| topic | Machine Learning Artificial Intelligence Human-Computer Interaction Methodology |
| url | https://arxiv.org/abs/2504.04243 |