Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.20956 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914582620012544 |
|---|---|
| author | Li, Chengze Liu, Xiao Zhang, Hanrong Peng, Haiyang Ruan, Yanghao Ma, Huanhuan Miao, Chunyu Zhou, Qichao Qi, Xiangrong Yu, Philip |
| author_facet | Li, Chengze Liu, Xiao Zhang, Hanrong Peng, Haiyang Ruan, Yanghao Ma, Huanhuan Miao, Chunyu Zhou, Qichao Qi, Xiangrong Yu, Philip |
| contents | Conformal triage converts predictive scores into deployment actions that either release a case, flag it for urgent attention, or defer it to human review. Under prevalence shift, however, the usual summaries of marginal coverage and human-review rate can miss the safety-critical question of whether patients who truly experience the target event are released without review. To address this gap, we introduce a leakage-aware deployment audit for release-side conformal triage. It first assigns target subjects to three non-overlapping roles: prevalence correction, conformal calibration, and held-out release-safety evaluation. This separation then lets the audit evaluate release directly: how many event-positive patients are cleared without review, whether the pilot has enough event labels for calibration, and how the safety-review trade-off shifts. Applying this audit to a retrospective NSCLC pilot shows why lower review can be misleading: after prevalence correction, the pooled conformal branch lowers review by releasing more patients, some of whom are event-positive. Within the audit, the classwise branch acts as a scarcity diagnostic: the pilot has too few event labels to certify safe low-review release. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_20956 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | A Deployment Audit of Release-Side Risk in Conformal Triage under Prevalence Shift Li, Chengze Liu, Xiao Zhang, Hanrong Peng, Haiyang Ruan, Yanghao Ma, Huanhuan Miao, Chunyu Zhou, Qichao Qi, Xiangrong Yu, Philip Machine Learning Computers and Society Conformal triage converts predictive scores into deployment actions that either release a case, flag it for urgent attention, or defer it to human review. Under prevalence shift, however, the usual summaries of marginal coverage and human-review rate can miss the safety-critical question of whether patients who truly experience the target event are released without review. To address this gap, we introduce a leakage-aware deployment audit for release-side conformal triage. It first assigns target subjects to three non-overlapping roles: prevalence correction, conformal calibration, and held-out release-safety evaluation. This separation then lets the audit evaluate release directly: how many event-positive patients are cleared without review, whether the pilot has enough event labels for calibration, and how the safety-review trade-off shifts. Applying this audit to a retrospective NSCLC pilot shows why lower review can be misleading: after prevalence correction, the pooled conformal branch lowers review by releasing more patients, some of whom are event-positive. Within the audit, the classwise branch acts as a scarcity diagnostic: the pilot has too few event labels to certify safe low-review release. |
| title | A Deployment Audit of Release-Side Risk in Conformal Triage under Prevalence Shift |
| topic | Machine Learning Computers and Society |
| url | https://arxiv.org/abs/2605.20956 |