Calibration plots for multistate risk prediction models
Fuente:
Wiley Open Access
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Artículo Open Access |
| Publié: |
Wiley
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1867019927396810752 |
|---|---|
| author | Alexander Pate Matthew Sperrin Richard D. Riley Niels Peek Tjeerd Van Staa Jamie C. Sergeant Mamas A. Mamas Gregory Y. H. Lip Martin O'Flaherty Michael Barrowman Iain Buchan Glen P. Martin |
| author_facet | Alexander Pate Matthew Sperrin Richard D. Riley Niels Peek Tjeerd Van Staa Jamie C. Sergeant Mamas A. Mamas Gregory Y. H. Lip Martin O'Flaherty Michael Barrowman Iain Buchan Glen P. Martin Alexander Pate Matthew Sperrin Richard D. Riley Niels Peek Tjeerd Van Staa Jamie C. Sergeant Mamas A. Mamas Gregory Y. H. Lip Martin O'Flaherty Michael Barrowman Iain Buchan Glen P. Martin |
| collection | Wiley Open Access |
| contents | Calibration plots for multistate risk prediction models Alexander Pate Matthew Sperrin Richard D. Riley Niels Peek Tjeerd Van Staa Jamie C. Sergeant Mamas A. Mamas Gregory Y. H. Lip Martin O'Flaherty Michael Barrowman Iain Buchan Glen P. Martin Statistics in Medicine IntroductionThere is currently no guidance on how to assess the calibration of multistate models used for risk prediction. We introduce several techniques that can be used to produce calibration plots for the transition probabilities of a multistate model, before assessing their performance in the presence of random and independent censoring through a simulation.MethodsWe studied pseudo‐values based on the Aalen‐Johansen estimator, binary logistic regression with inverse probability of censoring weights (BLR‐IPCW), and multinomial logistic regression with inverse probability of censoring weights (MLR‐IPCW). The MLR‐IPCW approach results in a calibration scatter plot, providing extra insight about the calibration. We simulated data with varying levels of censoring and evaluated the ability of each method to estimate the calibration curve for a set of predicted transition probabilities. We also developed and evaluated the calibration of a model predicting the incidence of cardiovascular disease, type 2 diabetes and chronic kidney disease among a cohort of patients derived from linked primary and secondary healthcare records.ResultsThe pseudo‐value, BLR‐IPCW, and MLR‐IPCW approaches give unbiased estimates of the calibration curves under random censoring. These methods remained predominately unbiased in the presence of independent censoring, even if the censoring mechanism was strongly associated with the outcome, with bias concentrated in low‐density regions of predicted transition probabilities.ConclusionsWe recommend implementing either the pseudo‐value or BLR‐IPCW approaches to produce a calibration curve, combined with the MLR‐IPCW approach to produce a calibration scatter plot. The methods have been incorporated into the “calibmsm” R package available on CRAN. 10.1002/sim.10094 http://creativecommons.org/licenses/by/4.0/ |
| doi_str_mv | 10.1002/sim.10094 |
| format | Artículo Open Access |
| id | wiley_oa_10_1002_sim_10094 |
| institution | Wiley Open Access |
| license_str_mv | http://creativecommons.org/licenses/by/4.0/ |
| publishDate | 2024 |
| publisher | Wiley |
| record_format | wiley_oa |
| spellingShingle | Calibration plots for multistate risk prediction models Alexander Pate Matthew Sperrin Richard D. Riley Niels Peek Tjeerd Van Staa Jamie C. Sergeant Mamas A. Mamas Gregory Y. H. Lip Martin O'Flaherty Michael Barrowman Iain Buchan Glen P. Martin Statistics in Medicine Calibration plots for multistate risk prediction models Alexander Pate Matthew Sperrin Richard D. Riley Niels Peek Tjeerd Van Staa Jamie C. Sergeant Mamas A. Mamas Gregory Y. H. Lip Martin O'Flaherty Michael Barrowman Iain Buchan Glen P. Martin Statistics in Medicine IntroductionThere is currently no guidance on how to assess the calibration of multistate models used for risk prediction. We introduce several techniques that can be used to produce calibration plots for the transition probabilities of a multistate model, before assessing their performance in the presence of random and independent censoring through a simulation.MethodsWe studied pseudo‐values based on the Aalen‐Johansen estimator, binary logistic regression with inverse probability of censoring weights (BLR‐IPCW), and multinomial logistic regression with inverse probability of censoring weights (MLR‐IPCW). The MLR‐IPCW approach results in a calibration scatter plot, providing extra insight about the calibration. We simulated data with varying levels of censoring and evaluated the ability of each method to estimate the calibration curve for a set of predicted transition probabilities. We also developed and evaluated the calibration of a model predicting the incidence of cardiovascular disease, type 2 diabetes and chronic kidney disease among a cohort of patients derived from linked primary and secondary healthcare records.ResultsThe pseudo‐value, BLR‐IPCW, and MLR‐IPCW approaches give unbiased estimates of the calibration curves under random censoring. These methods remained predominately unbiased in the presence of independent censoring, even if the censoring mechanism was strongly associated with the outcome, with bias concentrated in low‐density regions of predicted transition probabilities.ConclusionsWe recommend implementing either the pseudo‐value or BLR‐IPCW approaches to produce a calibration curve, combined with the MLR‐IPCW approach to produce a calibration scatter plot. The methods have been incorporated into the “calibmsm” R package available on CRAN. 10.1002/sim.10094 http://creativecommons.org/licenses/by/4.0/ |
| title | Calibration plots for multistate risk prediction models |
| topic | Statistics in Medicine |
| url | https://onlinelibrary.wiley.com/doi/10.1002/sim.10094 |