Nord, CL;
Valton, V;
Wood, J;
Roiser, JP;
(2017)
Power-up: a reanalysis of 'power failure' in neuroscience using mixture modelling.
Journal of Neuroscience
, 37
(34)
pp. 8051-8061.
10.1523/JNEUROSCI.3592-16.2017.
Preview |
Text
Nord_Power-up_reanalysis.pdf - Accepted Version Download (2MB) | Preview |
Abstract
Evidence for endemically low statistical power has recently cast neuroscience findings into doubt. If low statistical power plagues neuroscience, this reduces confidence in reported effects. However, if statistical power is not uniformly low, such blanket mistrust might not be warranted. Here, we provide a different perspective on this issue, analysing data from an influential paper reporting a median power of 21% across 49 meta-analyses (Button et al., 2013). We demonstrate, using Gaussian mixture modelling, that the sample of 730 studies included in that analysis comprises several subcomponents; therefore the use of a single summary statistic is insufficient to characterise the nature of the distribution. We find that statistical power is extremely low for studies included in meta-analyses that reported a null result; and that it varies substantially across subfields of neuroscience, with particularly low power in candidate gene association studies. Thus, while power in neuroscience remains a critical issue, the notion that studies are systematically underpowered is not the full story: low power is far from a universal problem. SIGNIFICANCE STATEMENT: Recently, researchers across the biomedical and psychological sciences have become concerned with the reliability of results. One marker for reliability is statistical power: the probability of finding a statistically significant result, given that the effect exists. Previous evidence suggests that statistical power is low across the field of neuroscience. Our results present a more comprehensive picture of statistical power in neuroscience: on average, studies are indeed underpowered-some very seriously so-but many studies show acceptable or even exemplary statistical power. We show that this heterogeneity in statistical power is common across most subfields in neuroscience (psychology, neuroimaging, etc.). This new, more nuanced picture of statistical power in neuroscience could affect not only scientific understanding, but potentially policy and funding decisions for neuroscience research.
Archive Staff Only
 |
View Item |
