Arno, Anneliese Downey;
(2022)
The Adoption and Effectiveness of Automation in Health Evidence Synthesis.
Doctoral thesis (Ph.D), UCL (University College London).
Preview |
Text
Arno - Automation in Health Evidence Synthesis.pdf - Accepted Version Download (4MB) | Preview |
Abstract
Background: Health systems worldwide are often informed by evidence-based guidelines which in turn rely heavily on systematic reviews. Systematic reviews are currently hindered by the increasing volume of new research and by its variable quality. Automation has potential to alleviate this problem but is not widely used in health evidence synthesis. This thesis sought to address the following: why is automation adopted (or not), and what effects does it have when it is put into use? / Methods: Roger’s Diffusion of Innovations theory, as a well-established and widely used framework, informed the study design and analysis. Adoption barriers and facilitators were explored through a thematic analysis of guideline developers’ opinions towards automation, and by mapping the adoption journey of a machine learning (ML) tool among Cochrane Information Specialists (CISs). A randomised trial of ML assistance in Risk of Bias (RoB) assessments and a cost-effectiveness analysis of a semi-automated workflow in the maintenance of a living evidence map each evaluated the effects of automation in practice. / Results: Adoption decisions are most strongly informed by the professional cultural expectations of health evidence synthesis. The stringent expectations of systematic reviewers and their users must be met before any other characteristic of an automation technology is considered by potential adopters. Ease-of-use increases in importance as a tool becomes more diffused across a population. Results of the randomised trial showed that ML-assisted RoB assessments were non-inferior to assessments completed entirely by human researcher effort. The cost-effectiveness analysis showed that a semi-automated workflow identified more relevant studies than the manual workflow and was less costly. / Conclusions: Automation can have substantial benefits when integrated into health evidence workflows. Wider adoption of automation tools will be facilitated by ensuring they are aligned with professional values of the field and limited in technical complexity.
| Type: | Thesis (Doctoral) |
|---|---|
| Qualification: | Ph.D |
| Title: | The Adoption and Effectiveness of Automation in Health Evidence Synthesis |
| Event: | UCL (University College London) |
| Open access status: | An open access version is available from UCL Discovery |
| Language: | English |
| Additional information: | Copyright © The Author 2022. Original content in this thesis is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) Licence (https://creativecommons.org/licenses/by-nc/4.0/). Any third-party copyright material present remains the property of its respective owner(s) and is licensed under its existing terms. Access may initially be restricted at the author’s request. |
| Keywords: | systematic reviews, automation, machine-learning, meta-analysis, public health |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Education UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education > IOE - Social Research Institute |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10141555 |
Archive Staff Only
 |
View Item |
