UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Are experiment sample sizes adequate to detect biologically important interactions between multiple stressors?

Burgess, Benjamin; Jackson, Michelle; Murrell, David; (2022) Are experiment sample sizes adequate to detect biologically important interactions between multiple stressors? bioRxiv Green open access

[thumbnail of Murrell_2021.07.21.453207v3.full.pdfpre.pdf]
Preview
Text
Murrell_2021.07.21.453207v3.full.pdfpre.pdf

Download (516kB) | Preview

Abstract

As most ecosystems are being challenged by multiple, co-occurring stressors, an important challenge is to understand and predict how stressors interact to affect biological responses. A popular approach is to design factorial experiments that measure biological responses to pairs of stressors and compare the observed response to a null model expectation. Unfortunately, we believe experiment sample sizes are inadequate to detect most non-null stressor interaction responses, greatly hindering progress. Determination of adequate sample size requires (i) knowledge of the detection ability of the inference method being used, and (ii) a consideration of the smallest biologically meaningful deviation from the null expectation. However, (i) has not been investigated and (ii) is yet to be discussed. Using both real and simulated data we show sample sizes typical of many experiments (<10) can only detect very large deviations from the additive null model, implying many important non-null stressor-pair interactions are being missed. We also highlight how only reporting statistically significant results at low samples sizes greatly overestimates the degree of non-additive stressor interactions. Computer code that simulates data under either additive or multiplicative null models is provided to estimate statistical power for user defined responses and sample sizes and we recommend this is used to aid experimental design and interpretation of results. We suspect that most experiments may require 20 or more replicates per treatment to have adequate power to detect non-additive. However, researchers still need to define the smallest interaction of interest, i.e. the lower limit for a biologically important interaction, which is likely to be system specific, meaning a general guide is unavailable. Sample sizes could potentially be increased by focussing on individual-level responses to multiple stressors, or by forming coordinated networks of researchers to repeat experiments in larger-scale studies. Our main analyses relate to the additive null model but we show similar problems occur for the multiplicative null model, and we encourage similar investigations into the statistical power of other null models and inference methods. Without knowledge of the detection abilities of the statistical tools at hand, or definition of the smallest meaningful interaction, we will undoubtedly continue to miss important ecosystem stressor interactions.

Type: Working / discussion paper
Title: Are experiment sample sizes adequate to detect biologically important interactions between multiple stressors?
Open access status: An open access version is available from UCL Discovery
DOI: 10.1101/2021.07.21.453207
Publisher version: https://doi.org/10.1101/2021.07.21.453207
Language: English
Additional information: The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC-ND 4.0 International license.
Keywords: Additive null model, critical effect size, ecosystem drivers, experimental design, minimum biological effect of interest, multiplicative null model, replicates, statistical power
UCL classification: UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Div of Biosciences > Genetics, Evolution and Environment
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Div of Biosciences
URI: https://discovery.ucl.ac.uk/id/eprint/10156112
Downloads since deposit
28Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item