UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Public availability and adherence to prespecified statistical analysis approaches was low in published randomized trials

Kahan, BC; Ahmad, T; Forbes, G; Cro, S; (2020) Public availability and adherence to prespecified statistical analysis approaches was low in published randomized trials. Journal of Clinical Epidemiology , 128 pp. 29-34. 10.1016/j.jclinepi.2020.07.015. Green open access

[thumbnail of Kahan_Public availability and adherence to pre-specified statistical analysis approaches was low in published randomised trials.pdf]
Preview
Text
Kahan_Public availability and adherence to pre-specified statistical analysis approaches was low in published randomised trials.pdf - Accepted Version

Download (639kB) | Preview

Abstract

BACKGROUND AND OBJECTIVE: Prespecification of statistical methods in clinical trial protocols and statistical analysis plans can help to deter bias from p-hacking but is only effective if the prespecified approach is made available. STUDY DESIGN AND SETTING: For 100 randomized trials published in 2018 and indexed in PubMed, we evaluated how often a prespecified statistical analysis approach for the trial's primary outcome was publicly available. For each trial with an available prespecified analysis, we compared this with the trial publication to identify whether there were unexplained discrepancies. RESULTS: Only 12 of 100 trials (12%) had a publicly available prespecified analysis approach for their primary outcome; this document was dated before recruitment began for only two trials. Of the 12 trials with an available prespecified analysis approach, 11 (92%) had one or more unexplained discrepancies. Only 4 of 100 trials (4%) stated that the statistician was blinded until the SAP was signed off, and only 10 of 100 (10%) stated the statistician was blinded until the database was locked. CONCLUSION: For most published trials, there is insufficient information available to determine whether the results may be subject to p-hacking. Where information was available, there were often unexplained discrepancies between the prespecified and final analysis methods.

Type: Article
Title: Public availability and adherence to prespecified statistical analysis approaches was low in published randomized trials
Location: United States
Open access status: An open access version is available from UCL Discovery
DOI: 10.1016/j.jclinepi.2020.07.015
Publisher version: https://doi.org/10.1016/j.jclinepi.2020.07.015
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Bias, P-hacking, Prespecification, Protocol, Randomized trial, Statistical analysis plan, Transparency
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Inst of Clinical Trials and Methodology
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Inst of Clinical Trials and Methodology > MRC Clinical Trials Unit at UCL
URI: https://discovery.ucl.ac.uk/id/eprint/10112813
Downloads since deposit
104Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item