UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

A Multicenter, Scan-Rescan, Human and Machine Learning CMR Study to Test Generalizability and Precision in Imaging Biomarker Analysis

Bhuva, AN; Bai, W; Lau, C; Davies, RH; Ye, Y; Bulluck, H; McAlindon, E; ... Manisty, CH; + view all (2019) A Multicenter, Scan-Rescan, Human and Machine Learning CMR Study to Test Generalizability and Precision in Imaging Biomarker Analysis. Circulation: Cardiovascular Imaging , 12 (10) , Article e009214. 10.1161/circimaging.119.009214. Green open access

[img]
Preview
Text
CIRCIMAGING.119.009214.pdf - Published version

Download (560kB) | Preview

Abstract

BACKGROUND: Automated analysis of cardiac structure and function using machine learning (ML) has great potential, but is currently hindered by poor generalizability. Comparison is traditionally against clinicians as a reference, ignoring inherent human inter- and intraobserver error, and ensuring that ML cannot demonstrate superiority. Measuring precision (scan:rescan reproducibility) addresses this. We compared precision of ML and humans using a multicenter, multi-disease, scan:rescan cardiovascular magnetic resonance data set. METHODS: One hundred ten patients (5 disease categories, 5 institutions, 2 scanner manufacturers, and 2 field strengths) underwent scan:rescan cardiovascular magnetic resonance (96% within one week). After identification of the most precise human technique, left ventricular chamber volumes, mass, and ejection fraction were measured by an expert, a trained junior clinician, and a fully automated convolutional neural network trained on 599 independent multicenter disease cases. Scan:rescan coefficient of variation and 1000 bootstrapped 95% CIs were calculated and compared using mixed linear effects models. RESULTS: Clinicians can be confident in detecting a 9% change in left ventricular ejection fraction, with greater than half of coefficient of variation attributable to intraobserver variation. Expert, trained junior, and automated scan:rescan precision were similar (for left ventricular ejection fraction, coefficient of variation 6.1 [5.2%–7.1%], P=0.2581; 8.3 [5.6%– 10.3%], P=0.3653; 8.8 [6.1%–11.1%], P=0.8620). Automated analysis was 186× faster than humans (0.07 versus 13 minutes). CONCLUSIONS: Automated ML analysis is faster with similar precision to the most precise human techniques, even when challenged with realworld scan:rescan data. Assessment of multicenter, multi-vendor, multifield strength scan:rescan data (available at www.thevolumesresource. com) permits a generalizable assessment of ML precision and may facilitate direct translation of ML to clinical practice.

Type: Article
Title: A Multicenter, Scan-Rescan, Human and Machine Learning CMR Study to Test Generalizability and Precision in Imaging Biomarker Analysis
Open access status: An open access version is available from UCL Discovery
DOI: 10.1161/circimaging.119.009214
Publisher version: https://doi.org/10.1161/circimaging.119.009214
Language: English
Additional information: This is an open access article under the terms of the Creative Commons Attribution Non-Commercial-NoDerivs License, which permits use, distribution, and reproduction in any medium, provided that the original work is properly cited, the use is noncommercial, and no modifications or adaptations are made https://creativecommons.org/licenses/by-nc-nd/3.0/
Keywords: artificial intelligence, image processing, left ventricular remodeling, magnetic resonance imaging, cine, ventricular function
UCL classification: UCL
UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Cardiovascular Science
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Cardiovascular Science > Clinical Science
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Cardiovascular Science > Population Science and Experimental Medicine
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Cardiovascular Science > Population Science and Experimental Medicine > MRC Unit for Lifelong Hlth and Ageing
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Health Informatics
URI: https://discovery.ucl.ac.uk/id/eprint/10082691
Downloads since deposit
135Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item