The expected value of sample information calculations for external validation of risk prediction models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909512878784512 |
|---|---|
| author | Sadatsafavi, Mohsen Vickers, Andrew J Lee, Tae Yoon Gustafson, Paul Wynants, Laure |
| author_facet | Sadatsafavi, Mohsen Vickers, Andrew J Lee, Tae Yoon Gustafson, Paul Wynants, Laure |
| contents | In designing external validation studies of clinical prediction models, contemporary sample size calculation methods are based on the frequentist inferential paradigm. One of the widely reported metrics of model performance is net benefit (NB), and the relevance of conventional inference around NB as a measure of clinical utility is doubtful. Value of Information methodology quantifies the consequences of uncertainty in terms of its impact on clinical utility of decisions. We introduce the expected value of sample information (EVSI) for validation as the expected gain in NB from conducting an external validation study of a given size. We propose algorithms for EVSI computation, and in a case study demonstrate how EVSI changes as a function of the amount of current information and future study's sample size. Value of Information methodology provides a decision-theoretic lens to the process of planning a validation study of a risk prediction model and can complement conventional methods when designing such studies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_01849 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The expected value of sample information calculations for external validation of risk prediction models Sadatsafavi, Mohsen Vickers, Andrew J Lee, Tae Yoon Gustafson, Paul Wynants, Laure Applications In designing external validation studies of clinical prediction models, contemporary sample size calculation methods are based on the frequentist inferential paradigm. One of the widely reported metrics of model performance is net benefit (NB), and the relevance of conventional inference around NB as a measure of clinical utility is doubtful. Value of Information methodology quantifies the consequences of uncertainty in terms of its impact on clinical utility of decisions. We introduce the expected value of sample information (EVSI) for validation as the expected gain in NB from conducting an external validation study of a given size. We propose algorithms for EVSI computation, and in a case study demonstrate how EVSI changes as a function of the amount of current information and future study's sample size. Value of Information methodology provides a decision-theoretic lens to the process of planning a validation study of a risk prediction model and can complement conventional methods when designing such studies. |
| title | The expected value of sample information calculations for external validation of risk prediction models |
| topic | Applications |
| url | https://arxiv.org/abs/2401.01849 |