Cheung, KKC;
Tai, KWH;
(2021)
The use of intercoder reliability in qualitative interview data analysis in science education.
Research in Science & Technological Education
10.1080/02635143.2021.1993179.
(In press).
Preview |
Text
The use of intercoder reliability in qualitative interview data analysis in science education.pdf - Published Version Download (2MB) | Preview |
Abstract
Background: Intercoder reliability is a statistic commonly reported by researchers to demonstrate the rigour of coding procedures during data analysis. Its importance is debatable in the analysis of qualitative interview data. It raises a question on whether researchers should identify the same codes and themes in a transcript or they should produce different accounts in analyzing the transcript. Purpose: This study reports how articles in four science education journals, International Journal of Science Education, Research in Science Education, Journal of Research in Science Teaching and Science Education report intercoder reliability in their analysis of interview data. Methods: This article explores whether 103 papers published in these science education journals in a single year (2019) have reported intercoder reliability test when the authors analyse their interview data. It was found that 19 papers have reported the test results. Findings: The authors of these studies have different interpretation towards a similar value of intercoder reliability. Moreover, the percentage of data used in the intercoder reliability test and the identity of intercoder vary across the studies. As a result, this paper aims to raise an issue on whether a replicability of coding can show the reliability of the results when researchers analyze interview data. Conclusion: We propose two major principles when authors report the reliability of the analysis of interview data: transparency and explanatory. We also argue that only when the authors report intercoder reliability test results that are based on these two principles, the reliability statistics of studies are convincing to readers. Some suggestions are offered to authors regarding how to carry out, analyze and report the intercoder reliability test.
| Type: | Article |
|---|---|
| Title: | The use of intercoder reliability in qualitative interview data analysis in science education |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1080/02635143.2021.1993179 |
| Publisher version: | https://doi.org/10.1080/02635143.2021.1993179 |
| Language: | English |
| Additional information: | © 2021 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. |
| Keywords: | Research methods, qualitative, intercoder reliability, percentage agreement, interview |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Education UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education > IOE - Centre for Doctoral Education |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10139100 |
Archive Staff Only
 |
View Item |
