Smith, Hayley;
Sweeting, Michael;
Morris, Tim;
Crowther, Michael J;
(2022)
A scoping methodological review of simulation studies comparing statistical and machine learning approaches to risk prediction for time-to-event data.
Diagnostic and Prognostic Research
, 6
(1)
, Article 10. 10.1186/s41512-022-00124-y.
Preview |
Text
A scoping methodological review of simulation studies comparing statistical and machine learning approaches to risk predicti.pdf - Published Version Download (1MB) | Preview |
Abstract
BACKGROUND: There is substantial interest in the adaptation and application of so-called machine learning approaches to prognostic modelling of censored time-to-event data. These methods must be compared and evaluated against existing methods in a variety of scenarios to determine their predictive performance. A scoping review of how machine learning methods have been compared to traditional survival models is important to identify the comparisons that have been made and issues where they are lacking, biased towards one approach or misleading. METHODS: We conducted a scoping review of research articles published between 1 January 2000 and 2 December 2020 using PubMed. Eligible articles were those that used simulation studies to compare statistical and machine learning methods for risk prediction with a time-to-event outcome in a medical/healthcare setting. We focus on data-generating mechanisms (DGMs), the methods that have been compared, the estimands of the simulation studies, and the performance measures used to evaluate them. RESULTS: A total of ten articles were identified as eligible for the review. Six of the articles evaluated a method that was developed by the authors, four of which were machine learning methods, and the results almost always stated that this developed method's performance was equivalent to or better than the other methods compared. Comparisons were often biased towards the novel approach, with the majority only comparing against a basic Cox proportional hazards model, and in scenarios where it is clear it would not perform well. In many of the articles reviewed, key information was unclear, such as the number of simulation repetitions and how performance measures were calculated. CONCLUSION: It is vital that method comparisons are unbiased and comprehensive, and this should be the goal even if realising it is difficult. Fully assessing how newly developed methods perform and how they compare to a variety of traditional statistical methods for prognostic modelling is imperative as these methods are already being applied in clinical contexts. Evaluations of the performance and usefulness of recently developed methods for risk prediction should be continued and reporting standards improved as these methods become increasingly popular.
| Type: | Article |
|---|---|
| Title: | A scoping methodological review of simulation studies comparing statistical and machine learning approaches to risk prediction for time-to-event data |
| Location: | England |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1186/s41512-022-00124-y |
| Publisher version: | https://doi.org/10.1186/s41512-022-00124-y |
| Language: | English |
| Additional information: | © The Author(s). 2022 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. |
| Keywords: | Clinical risk prediction, Machine learning, Prognostic modelling, Simulation studies, Survival analysis |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Inst of Clinical Trials and Methodology > MRC Clinical Trials Unit at UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Inst of Clinical Trials and Methodology UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10150281 |
Archive Staff Only
 |
View Item |
