Pavlou, M;
Qu, C;
Omar, RZ;
Seaman, SR;
Steyerberg, EW;
White, IR;
Ambler, G;
(2021)
Estimation of required sample size for external validation of risk models for binary outcomes.
Statistical Methods in Medical Research
10.1177/09622802211007522.
(In press).
Preview |
Text
Pavlou_Estimation of required sample size for external validation of risk models for binary outcomes_AOP.pdf - Published Version Download (520kB) | Preview |
Abstract
Risk-prediction models for health outcomes are used in practice as part of clinical decision-making, and it is essential that their performance be externally validated. An important aspect in the design of a validation study is choosing an adequate sample size. In this paper, we investigate the sample size requirements for validation studies with binary outcomes to estimate measures of predictive performance (C-statistic for discrimination and calibration slope and calibration in the large). We aim for sufficient precision in the estimated measures. In addition, we investigate the sample size to achieve sufficient power to detect a difference from a target value. Under normality assumptions on the distribution of the linear predictor, we obtain simple estimators for sample size calculations based on the measures above. Simulation studies show that the estimators perform well for common values of the C-statistic and outcome prevalence when the linear predictor is marginally Normal. Their performance deteriorates only slightly when the normality assumptions are violated. We also propose estimators which do not require normality assumptions but require specification of the marginal distribution of the linear predictor and require the use of numerical integration. These estimators were also seen to perform very well under marginal normality. Our sample size equations require a specified standard error (SE) and the anticipated C-statistic and outcome prevalence. The sample size requirement varies according to the prognostic strength of the model, outcome prevalence, choice of the performance measure and study objective. For example, to achieve an SE < 0.025 for the C-statistic, 60-170 events are required if the true C-statistic and outcome prevalence are between 0.64-0.85 and 0.05-0.3, respectively. For the calibration slope and calibration in the large, achieving SE < 0.15 would require 40-280 and 50-100 events, respectively. Our estimators may also be used for survival outcomes when the proportion of censored observations is high.
| Type: | Article |
|---|---|
| Title: | Estimation of required sample size for external validation of risk models for binary outcomes |
| Location: | England |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1177/09622802211007522 |
| Publisher version: | https://doi.org/10.1177%2F09622802211007522 |
| Language: | English |
| Additional information: | © 2021 by SAGE Publications. This article is distributed under the terms of the Creative Commons Attribution 4.0 License (https://creativecommons.org/licenses/by/4.0/). |
| Keywords: | C-statistic, Sample size calculation, calibration, discrimination, prediction model |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Inst of Clinical Trials and Methodology UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences > Dept of Statistical Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10126533 |
Archive Staff Only
 |
View Item |
