UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

The effects of mixed mode survey designs on simple and complex analyses

Martin, P; Lynn, P; (2011) The effects of mixed mode survey designs on simple and complex analyses. (Centre for Comparative Social Surveys Working Paper Series 04). Centre for Comparative Social Surveys, City University London: London, UK. Green open access

[thumbnail of Martin & Lynn (2011) The effect of mixed mode survey design on simple and complex analyses.pdf]
Preview
Text
Martin & Lynn (2011) The effect of mixed mode survey design on simple and complex analyses.pdf

Download (367kB) | Preview

Abstract

Survey data can be collected in a number of ways. The survey organisation may use face-toface interviewing or telephone interviewing, or may ask respondents to complete questionnaires themselves, either online or on paper. It is becoming increasingly common for surveys to use a combination of these methods, a so-called “mixed mode” design. However, the choice of mode, or mix of modes, can affect the data that are collected and consequently also the estimates that are made based on those data. This can happen because different kinds of people are more likely to participate in different modes, or because people will give slightly different answers depending on the mode of interviewing. In this paper we compare estimates from surveys that used three different designs. One survey was a single mode survey, carried out entirely through face-to-face interviewing. The other two surveys were mixed mode surveys, involving face-to-face and telephone interviewing as well as online (web) questionnaires. The difference between the two mixed mode surveys was that in one case respondents were offered an explicit choice between the three interview modes, while in the other case respondents were first asked to complete a web questionnaire and only if they were unable or unwilling to do so were they then asked instead for a telephone interview. If the respondent was also unable or unwilling to carry out the telephone interview they were then asked for a face-to-face interview. Our main interest lies in comparing each of the mixed mode designs with the single mode design. An identical questionnaire was administered to each of the three samples, namely the round 4 questionnaire of the European Social Survey. We compare the distributions of answers to each survey question, and we also compare some estimates of regression coefficients from statistical models of the kind often used by political scientists. While many estimates show no significant difference between the survey designs, we conclude that some estimates are affected by the survey design. We find some suggestive evidence that this is likely to be, at least partly, due to differences in how people answer questions in different modes rather than due to different types of people taking part in the survey. We consequently urge caution in comparing estimates based on data collected using different (mixes of) modes and in the use of mixed mode survey designs.

Type: Working / discussion paper
Title: The effects of mixed mode survey designs on simple and complex analyses
Open access status: An open access version is available from UCL Discovery
Publisher version: https://www.city.ac.uk/__data/assets/pdf_file/0017...
Language: English
Additional information: This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Epidemiology and Health
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Epidemiology and Health > Applied Health Research
URI: https://discovery.ucl.ac.uk/id/eprint/1551618
Downloads since deposit
21Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item