UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

A machine learning system to optimise triage in an adult ophthalmic emergency department: a model development and validation study

Brandao-de-Resende, C; Melo, M; Lee, E; Jindal, A; Neo, YN; Sanghi, P; Freitas, JR; ... Day, AC; + view all (2023) A machine learning system to optimise triage in an adult ophthalmic emergency department: a model development and validation study. eClinicalMedicine , 66 , Article 102331. 10.1016/j.eclinm.2023.102331. Green open access

[thumbnail of 1-s2.0-S2589537023005084-main.pdf]
Preview
Text
1-s2.0-S2589537023005084-main.pdf - Published Version

Download (841kB) | Preview

Abstract

BACKGROUND: A substantial proportion of attendances to ophthalmic emergency departments are for non-urgent presentations. We developed and evaluated a machine learning system (DemDx Ophthalmology Triage System: DOTS) to optimise triage, with the aim of reducing inappropriate emergency attendances and streamlining case referral when necessary. METHODS: DOTS was built using retrospective tabular data from 11,315 attendances between July 1st, 2021, to June 15th, 2022 at Moorfields Eye Hospital Emergency Department (MEH) in London, UK. Demographic and clinical features were used as inputs and a triage recommendation was given (“see immediately”, “see within a week”, or “see electively”). DOTS was validated temporally and compared with triage nurses’ performance (1269 attendances at MEH) and validated externally (761 attendances at the Federal University of Minas Gerais - UFMG, Brazil). It was also tested for biases and robustness to variations in disease incidences. All attendances from patients aged at least 18 years with at least one confirmed diagnosis were included in the study. FINDINGS: For identifying ophthalmic emergency attendances, on temporal validation, DOTS had a sensitivity of 94.5% [95% CI 92.3–96.1] and a specificity of 42.4% [38.8–46.1]. For comparison within the same dataset, triage nurses had a sensitivity of 96.4% [94.5–97.7] and a specificity of 25.1% [22.0–28.5]. On external validation at UFMG, DOTS had a sensitivity of 95.2% [92.5–97.0] and a specificity of 32.2% [27.4–37.0]. In simulated scenarios with varying disease incidences, the sensitivity was ≥92.2% and the specificity was ≥36.8%. No differences in sensitivity were found in subgroups of index of multiple deprivation, but the specificity was higher for Q2 when compared to Q4 (Q4 is less deprived than Q2). INTERPRETATION: At MEH, DOTS had similar sensitivity to triage nurses in determining attendance priority; however, with a specificity of 17.3% higher, DOTS resulted in lower rates of patients triaged to be seen immediately at emergency. DOTS showed consistent performance in temporal and external validation, in social-demographic subgroups and was robust to varying relative disease incidences. Further trials are necessary to validate these findings. This system will be prospectively evaluated, considering human-computer interaction, in a clinical trial. FUNDING: The Artificial Intelligence in Health and Care Award (AI_AWARD01671) of the NHS AI Lab under National Institute for Health and Care Research (NIHR) and the Accelerated Access Collaborative (AAC).

Type: Article
Title: A machine learning system to optimise triage in an adult ophthalmic emergency department: a model development and validation study
Location: England
Open access status: An open access version is available from UCL Discovery
DOI: 10.1016/j.eclinm.2023.102331
Publisher version: https://doi.org/10.1016/j.eclinm.2023.102331
Language: English
Additional information: © 2023 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Keywords: Artificial intelligence, Digital health, Emergency care, Machine learning, Ophthalmology, Triage
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Institute of Ophthalmology
URI: https://discovery.ucl.ac.uk/id/eprint/10184306
Downloads since deposit
4Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item