UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Treating Words as Data with Error: Uncertainty in Text Statements of Policy Positions

Benoit, K; Laver, M; Mikhaylov, S; (2009) Treating Words as Data with Error: Uncertainty in Text Statements of Policy Positions. American Journal of Political Science , 53 (2) pp. 495-513. 10.1111/j.1540-5907.2009.00383.x. Green open access

[thumbnail of Mikhaylov_CMP_AJPS3.pdf]
Preview
Text
Mikhaylov_CMP_AJPS3.pdf

Download (683kB) | Preview

Abstract

Political text offers extraordinary potential as a source of information about the policy positions of political actors. Despite recent advances in computational text analysis, human interpretative coding of text remains an important source of text-based data, ultimately required to validate more automatic techniques. The profession's main source of cross-national, time-series data on party policy positions comes from the human interpretative coding of party manifestos by the Comparative Manifesto Project (CMP). Despite widespread use of these data, the uncertainty associated with each point estimate has never been available, undermining the value of the dataset as a scientific resource. We propose a remedy. First, we characterize processes by which CMP data are generated. These include inherently stochastic processes of text authorship, as well as of the parsing and coding of observed text by humans. Second, we simulate these error-generating processes by bootstrapping analyses of coded quasi-sentences. This allows us to estimate precise levels of nonsystematic error for every category and scale reported by the CMP for its entire set of 3,000-plus manifestos. Using our estimates of these errors, we show how to correct biased inferences, in recent prominently published work, derived from statistical analyses of error-contaminated CMP data.

Type: Article
Title: Treating Words as Data with Error: Uncertainty in Text Statements of Policy Positions
Open access status: An open access version is available from UCL Discovery
DOI: 10.1111/j.1540-5907.2009.00383.x
Publisher version: http://dx.doi.org/10.1111/j.1540-5907.2009.00383.x
Language: English
Additional information: This is the peer reviewed version of the following article: Benoit, K; Laver, M; Mikhaylov, S; (2009) Treating Words as Data with Error: Uncertainty in Text Statements of Policy Positions. American Journal of Political Science, 53 (2) pp. 495-513, which has been published in final form at: 10.1111/j.1540-5907.2009.00383.x. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving (http://olabout.wiley.com/WileyCDA/Section/id-828039.html#terms).
Keywords: POLITICAL TEXTS, PARTIES
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL SLASH
UCL > Provost and Vice Provost Offices > UCL SLASH > Faculty of S&HS
UCL > Provost and Vice Provost Offices > UCL SLASH > Faculty of S&HS > Dept of Political Science
URI: https://discovery.ucl.ac.uk/id/eprint/135355
Downloads since deposit
418Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item