Designing a quasi-experiment to study the clinical impact of adaptive risk prediction models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914123394056192 |
|---|---|
| author | Odeh-Couvertier, Valerie Zayas-Caban, Gabriel Patterson, Brian Cochran, Amy |
| author_facet | Odeh-Couvertier, Valerie Zayas-Caban, Gabriel Patterson, Brian Cochran, Amy |
| contents | Clinical risk prediction is a valuable tool for guiding healthcare interventions toward those most likely to benefit. Yet, evaluating the pairing of a risk prediction model with an intervention using randomized controlled trials presents substantial challenges, making quasi-experimental designs an attractive alternatives. Existing designs, however, assume that both the model and the decision rules used to trigger interventions (typically a risk threshold) remain fixed. This limits their utility in modern healthcare, where both are routinely updated. We introduce a regression discontinuity framework that accommodates adaptation in both the model and the risk threshold. We precisely characterize the form of interference introduced by these adaptations and exploit this structure to establish conditions for identification and thus design estimation strategies. The key idea is to define counterfactual risks-the scores patients would have received under hypothetical reorderings-thereby restoring local exchangeability and enabling valid estimation of the local average treatment effect. Our estimator leverages the fact that, although counterfactual risk vectors grow with time, they typically lie in a low-dimensional space. In simulations of cardiovascular prevention programs, we show that our method accurately recovers treatment effects even as thresholds adapt to meet operational or clinical targets and models are updated to align predicted and observed outcomes or to exclude demographic predictors such as race. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_25052 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Designing a quasi-experiment to study the clinical impact of adaptive risk prediction models Odeh-Couvertier, Valerie Zayas-Caban, Gabriel Patterson, Brian Cochran, Amy Methodology Clinical risk prediction is a valuable tool for guiding healthcare interventions toward those most likely to benefit. Yet, evaluating the pairing of a risk prediction model with an intervention using randomized controlled trials presents substantial challenges, making quasi-experimental designs an attractive alternatives. Existing designs, however, assume that both the model and the decision rules used to trigger interventions (typically a risk threshold) remain fixed. This limits their utility in modern healthcare, where both are routinely updated. We introduce a regression discontinuity framework that accommodates adaptation in both the model and the risk threshold. We precisely characterize the form of interference introduced by these adaptations and exploit this structure to establish conditions for identification and thus design estimation strategies. The key idea is to define counterfactual risks-the scores patients would have received under hypothetical reorderings-thereby restoring local exchangeability and enabling valid estimation of the local average treatment effect. Our estimator leverages the fact that, although counterfactual risk vectors grow with time, they typically lie in a low-dimensional space. In simulations of cardiovascular prevention programs, we show that our method accurately recovers treatment effects even as thresholds adapt to meet operational or clinical targets and models are updated to align predicted and observed outcomes or to exclude demographic predictors such as race. |
| title | Designing a quasi-experiment to study the clinical impact of adaptive risk prediction models |
| topic | Methodology |
| url | https://arxiv.org/abs/2510.25052 |