UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Health record hiccups—5,526 real-world time series with change points labelled by crowdsourced visual inspection

Quan, T Phuong; Lacey, Ben; Peto, Tim EA; Walker, A Sarah; (2023) Health record hiccups—5,526 real-world time series with change points labelled by crowdsourced visual inspection. GigaScience , 12 , Article giad060. 10.1093/gigascience/giad060. Green open access

[thumbnail of Walker_giad060.pdf]
Preview
Text
Walker_giad060.pdf

Download (2MB) | Preview

Abstract

Background: Large routinely collected data such as electronic health records (EHRs) are increasingly used in research, but the statistical methods and processes used to check such data for temporal data quality issues have not moved beyond manual, ad hoc production and visual inspection of graphs. With the prospect of EHR data being used for disease surveillance via automated pipelines and public-facing dashboards, automation of data quality checks will become increasingly valuable. / Findings: We generated 5,526 time series from 8 different EHR datasets and engaged >2,000 citizen-science volunteers to label the locations of all suspicious-looking change points in the resulting graphs. Consensus labels were produced using density-based clustering with noise, with validation conducted using 956 images containing labels produced by an experienced data scientist. Parameter tuning was done against 670 images and performance calculated against 286 images, resulting in a final sensitivity of 80.4% (95% CI, 77.1%–83.3%), specificity of 99.8% (99.7%–99.8%), positive predictive value of 84.5% (81.4%–87.2%), and negative predictive value of 99.7% (99.6%–99.7%). In total, 12,745 change points were found within 3,687 of the time series. / Conclusions: This large collection of labelled EHR time series can be used to validate automated methods for change point detection in real-world settings, encouraging the development of methods that can successfully be applied in practice. It is particularly valuable since change point detection methods are typically validated using synthetic data, so their performance in real-world settings cannot be assumed to be comparable. While the dataset focusses on EHRs and data quality, it should also be applicable in other fields.

Type: Article
Title: Health record hiccups—5,526 real-world time series with change points labelled by crowdsourced visual inspection
Open access status: An open access version is available from UCL Discovery
DOI: 10.1093/gigascience/giad060
Publisher version: https://doi.org/10.1093/gigascience/giad060
Language: English
Additional information: Copyright © The Author(s) 2023. Published by Oxford University Press GigaScience. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Inst of Clinical Trials and Methodology
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Inst of Clinical Trials and Methodology > MRC Clinical Trials Unit at UCL
URI: https://discovery.ucl.ac.uk/id/eprint/10173053
Downloads since deposit
5Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item