Towler, Lauren;
Bondaronek, Paulina;
Papakonstantinou, Trisevgeni;
Amlot, Richard;
Chadborn, Tim;
Ainsworth, Ben;
Yardley, Lucy;
(2023)
Applying machine-learning to rapidly analyze large qualitative text datasets to inform the COVID-19 pandemic response: comparing human and machine-assisted topic analysis techniques.
Frontiers in Public Health
, 11
, Article 1268223. 10.3389/fpubh.2023.1268223.
Preview |
PDF
fpubh-11-1268223.pdf - Published Version Download (1MB) | Preview |
Abstract
Introduction: Machine-assisted topic analysis (MATA) uses artificial intelligence methods to help qualitative researchers analyze large datasets. This is useful for researchers to rapidly update healthcare interventions during changing healthcare contexts, such as a pandemic. We examined the potential to support healthcare interventions by comparing MATA with “human-only” thematic analysis techniques on the same dataset (1,472 user responses from a COVID-19 behavioral intervention). Methods: In MATA, an unsupervised topic-modeling approach identified latent topics in the text, from which researchers identified broad themes. In human-only codebook analysis, researchers developed an initial codebook based on previous research that was applied to the dataset by the team, who met regularly to discuss and refine the codes. Formal triangulation using a “convergence coding matrix” compared findings between methods, categorizing them as “agreement”, “complementary”, “dissonant”, or “silent”. Results: Human analysis took much longer than MATA (147.5 vs. 40 h). Both methods identified key themes about what users found helpful and unhelpful. Formal triangulation showed both sets of findings were highly similar. The formal triangulation showed high similarity between the findings. All MATA codes were classified as in agreement or complementary to the human themes. When findings differed slightly, this was due to human researcher interpretations or nuance from human-only analysis. Discussion: Results produced by MATA were similar to human-only thematic analysis, with substantial time savings. For simple analyses that do not require an in-depth or subtle understanding of the data, MATA is a useful tool that can support qualitative researchers to interpret and analyze large datasets quickly. This approach can support intervention development and implementation, such as enabling rapid optimization during public health emergencies.
Type: | Article |
---|---|
Title: | Applying machine-learning to rapidly analyze large qualitative text datasets to inform the COVID-19 pandemic response: comparing human and machine-assisted topic analysis techniques |
Location: | Switzerland |
Open access status: | An open access version is available from UCL Discovery |
DOI: | 10.3389/fpubh.2023.1268223 |
Publisher version: | http://dx.doi.org/10.3389/fpubh.2023.1268223 |
Language: | English |
Additional information: | This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third-party material in this article are included in the Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ |
Keywords: | Science & Technology, Life Sciences & Biomedicine, Public, Environmental & Occupational Health, public health, interventions, qualitative analysis, machine learning techniques, triangulation |
UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Health Informatics UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Health Informatics > CHIME |
URI: | https://discovery.ucl.ac.uk/id/eprint/10191745 |
Archive Staff Only
View Item |