Heyman, Tom;
Pronizius, Ekaterina;
Lewis, Savannah C;
Acar, Oguz A;
Adamkovič, Matúš;
Ambrosini, Ettore;
Antfolk, Jan;
... Buchanan, Erin M; + view all
(2025)
Crowdsourcing multiverse analyses to explore the impact of different data-processing and analysis decisions: A tutorial.
Psychological Methods
10.1037/met0000770.
(In press).
Preview |
Text
Manuscript.pdf - Accepted Version Download (1MB) | Preview |
Abstract
When processing and analyzing empirical data, researchers regularly face choices that may appear arbitrary (e.g., how to define and handle outliers). If one chooses to exclusively focus on a particular option and conduct a single analysis, its outcome might be of limited utility. That is, one remains agnostic regarding the generalizability of the results, because plausible alternative paths remain unexplored. A multiverse analysis offers a solution to this issue by exploring the various choices pertaining to data-processing and/or model building, and examining their impact on the conclusion of a study. However, even though multiverse analyses are arguably less susceptible to biases compared to the typical single-pathway approach, it is still possible to selectively add or omit pathways. To address this issue, we outline a novel, more principled approach to conducting multiverse analyses through crowdsourcing. The approach is detailed in a step-by-step tutorial to facilitate its implementation. We also provide a worked-out illustration featuring the Semantic Priming Across Many Languages project, thereby demonstrating its feasibility and its ability to increase objectivity and transparency.
Archive Staff Only
 |
View Item |
