Bondaronek, P;
Papakonstantinou, T;
Stefanidou, C;
Chadborn, T;
(2023)
User feedback on the NHS test & Trace Service during COVID-19: The use of machine learning to analyse free-text data from 37,914 England adults.
Public Health in Practice
, 6
, Article 100401. 10.1016/j.puhip.2023.100401.
Preview |
PDF
1-s2.0-S2666535223000472-main.pdf - Published Version Download (2MB) | Preview |
Abstract
Objectives: The UK government's approach to the pandemic relies on a test, trace and isolate strategy, mainly implemented via the digital NHS Test & Trace Service. Feedback on user experience is central to the successful development of public-facing Services. As the situation dynamically changes and data accumulate, interpretation of feedback by humans becomes time-consuming and unreliable. The specific objectives were to 1) evaluate a human-in-the-loop machine learning technique based on structural topic modelling in terms of its Service ability in the analysis of vast volumes of free-text data, 2) generate actionable themes that can be used to increase user satisfaction of the Service. Methods: We evaluated an unsupervised Topic Modelling approach, testing models with 5–40 topics and differing covariates. Two human coders conducted thematic analysis to interpret the topics. We identified a Structural Topic Model with 25 topics and metadata as covariates as the most appropriate for acquiring insights. Results: Results from analysis of feedback by 37,914 users from May 2020 to March 2021 highlighted issues with the Service falling within three major themes: multiple contacts and incompatible contact method and incompatible contact method, confusion around isolation dates and tracing delays, complex and rigid system. Conclusions: Structural Topic Modelling coupled with thematic analysis was found to be an effective technique to rapidly acquire user insights. Topic modelling can be a quick and cost-effective method to provide high quality, actionable insights from free-text feedback to optimize public health Services.
| Type: | Article |
|---|---|
| Title: | User feedback on the NHS test & Trace Service during COVID-19: The use of machine learning to analyse free-text data from 37,914 England adults |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1016/j.puhip.2023.100401 |
| Publisher version: | http://dx.doi.org/10.1016/j.puhip.2023.100401 |
| Language: | English |
| Additional information: | This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). |
| Keywords: | Science & Technology, Life Sciences & Biomedicine, Public, Environmental & Occupational Health, Public health, Machine learning, Qualitative data, Contact tracing, COVID-19 |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Health Informatics UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Health Informatics > CHIME |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10191741 |
Archive Staff Only
 |
View Item |
