UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Crowd-sourced Text Analysis: Reproducible and Agile Production of Political Data

Benoit, K; Conway, D; Lauderdale, BE; Laver, M; Mikhaylov, S; (2016) Crowd-sourced Text Analysis: Reproducible and Agile Production of Political Data. American Political Science Review , 110 (2) pp. 278-295. 10.1017/S0003055416000058. Green open access

[thumbnail of Crowd-sourced_text_anlaysis.pdf]
Preview
Text
Crowd-sourced_text_anlaysis.pdf - Accepted Version

Download (2MB) | Preview

Abstract

Empirical social science often relies on data that are not observed in the field, but are transformed into quantitative variables by expert researchers who analyze and interpret qualitative raw sources. While generally considered the most valid way to produce data, this expert-driven process is inherently difficult to replicate or to assess on grounds of reliability. Using crowd-sourcing to distribute text for reading and interpretation by massive numbers of nonexperts, we generate results comparable to those using experts to read and interpret the same texts, but do so far more quickly and flexibly. Crucially, the data we collect can be reproduced and extended transparently, making crowd-sourced datasets intrinsically reproducible. This focuses researchers’ attention on the fundamental scientific objective of specifying reliable and replicable methods for collecting the data needed, rather than on the content of any particular dataset. We also show that our approach works straightforwardly with different types of political text, written in different languages. While findings reported here concern text analysis, they have far-reaching implications for expert-generated data in the social sciences.

Type: Article
Title: Crowd-sourced Text Analysis: Reproducible and Agile Production of Political Data
Open access status: An open access version is available from UCL Discovery
DOI: 10.1017/S0003055416000058
Publisher version: http://doi.org/10.1017/S0003055416000058
Language: English
Additional information: Copyright © American Political Science Association 2017. All rights reserved. This article has been published in a revised form in American Political Science Review http://doi.org/10.1017/S0003055416000058. This version is free to view and download for private research and study only. Not for re-distribution, re-sale or use in derivative works. © American Political Science Association 2017.
Keywords: Social Sciences, Political Science, Government & Law, Amazon Mechanical Turk, Party, Reliability
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL SLASH
UCL > Provost and Vice Provost Offices > UCL SLASH > Faculty of S&HS
UCL > Provost and Vice Provost Offices > UCL SLASH > Faculty of S&HS > Dept of Political Science
URI: https://discovery.ucl.ac.uk/id/eprint/1508406
Downloads since deposit
87Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item