Takada, T;
Nijman, S;
Denaxas, S;
Snell, KIE;
Uijl, A;
Nguyen, T-L;
Asselbergs, FW;
(2021)
Internal-external cross-validation helped to evaluate the generalizability of prediction models in large clustered datasets.
Journal of Clinical Epidemiology
10.1016/j.jclinepi.2021.03.025.
(In press).
Preview |
Text
1-s2.0-S0895435621001074-main.pdf - Published Version Download (647kB) | Preview |
Abstract
OBJECTIVE: To illustrate how to evaluate the need of complex strategies for developing generalizable prediction models in large clustered datasets. STUDY DESIGN AND SETTING: We developed eight Cox regression models to estimate the risk of heart failure using a large population-level dataset. These models differed in the number of predictors, the functional form of the predictor effects (non-linear effects and interaction) and the estimation method (maximum likelihood and penalization). Internal-external cross-validation was used to evaluate the models' generalizability across the included general practices. RESULTS: Among 871,687 individuals from 225 general practices, 43,987 (5.5%) developed heart failure during a median follow-up time of 5.8 years. For discrimination, the simplest prediction model yielded a good concordance statistic, which was not much improved by adopting complex strategies. Between-practice heterogeneity in discrimination was similar in all models. For calibration, the simplest model performed satisfactorily. Although accounting for non-linear effects and interaction slightly improved the calibration slope, it also led to more heterogeneity in the observed/expected ratio. Similar results were found in a second case study involving patients with stroke. CONCLUSION: In large clustered datasets, prediction model studies may adopt internal-external cross-validation to evaluate the generalizability of competing models, and to identify promising modelling strategies.
| Type: | Article |
|---|---|
| Title: | Internal-external cross-validation helped to evaluate the generalizability of prediction models in large clustered datasets |
| Location: | United States |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1016/j.jclinepi.2021.03.025 |
| Publisher version: | https://doi.org/10.1016/j.jclinepi.2021.03.025 |
| Language: | English |
| Additional information: | © 2021 The Authors. Published by Elsevier Inc. under a Creative Commons license (https://creativecommons.org/licenses/by/4.0/). |
| Keywords: | Calibration, Discrimination, Heterogeneity, Model comparison, Prediction model, Validation |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Health Informatics UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Health Informatics > Clinical Epidemiology |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10126759 |
Archive Staff Only
 |
View Item |
