Efthimiou, Orestis;
Hoogland, Jeroen;
Debray, Thomas PA;
Seo, Michael;
Furukawa, Toshiaki A;
Egger, Matthias;
White, Ian R;
(2023)
Measuring the performance of prediction models to personalize treatment choice.
Statistics in Medicine
10.1002/sim.9665.
(In press).
Preview |
PDF
Statistics in Medicine - 2023 - Efthimiou - Measuring the performance of prediction models to personalize treatment choice.pdf - Published Version Download (2MB) | Preview |
Abstract
When data are available from individual patients receiving either a treatment or a control intervention in a randomized trial, various statistical and machine learning methods can be used to develop models for predicting future outcomes under the two conditions, and thus to predict treatment effect at the patient level. These predictions can subsequently guide personalized treatment choices. Although several methods for validating prediction models are available, little attention has been given to measuring the performance of predictions of personalized treatment effect. In this article, we propose a range of measures that can be used to this end. We start by defining two dimensions of model accuracy for treatment effects, for a single outcome: discrimination for benefit and calibration for benefit. We then amalgamate these two dimensions into an additional concept, decision accuracy, which quantifies the model's ability to identify patients for whom the benefit from treatment exceeds a given threshold. Subsequently, we propose a series of performance measures related to these dimensions and discuss estimating procedures, focusing on randomized data. Our methods are applicable for continuous or binary outcomes, for any type of prediction model, as long as it uses baseline covariates to predict outcomes under treatment and control. We illustrate all methods using two simulated datasets and a real dataset from a trial in depression. We implement all methods in the R package predieval. Results suggest that the proposed measures can be useful in evaluating and comparing the performance of competing models in predicting individualized treatment effect.
| Type: | Article |
|---|---|
| Title: | Measuring the performance of prediction models to personalize treatment choice |
| Location: | England |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1002/sim.9665 |
| Publisher version: | https://doi.org/10.1002/sim.9665 |
| Language: | English |
| Additional information: | © 2023 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd. This is an open access article under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/). |
| Keywords: | heterogeneous treatment effects, personalized medicine, prediction modelling |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Inst of Clinical Trials and Methodology |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10164082 |
Archive Staff Only
 |
View Item |
