UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Voice-assisted Image Labelling for Endoscopic Ultrasound Classification using Neural Networks

Bonmati, E; Hu, Y; Grimwood, A; Johnson, GJ; Goodchild, G; Keane, MG; Gurusamy, K; ... Barratt, DC; + view all (2021) Voice-assisted Image Labelling for Endoscopic Ultrasound Classification using Neural Networks. IEEE Transactions on Medical Imaging 10.1109/TMI.2021.3139023. (In press). Green open access

[thumbnail of Voice-assisted_Image_Labelling_for_Endoscopic_Ultrasound_Classification_using_Neural_Networks.pdf]
Preview
Text
Voice-assisted_Image_Labelling_for_Endoscopic_Ultrasound_Classification_using_Neural_Networks.pdf - Accepted Version

Download (1MB) | Preview

Abstract

Ultrasound imaging is a commonly used technology for visualising patient anatomy in real-time during diagnostic and therapeutic procedures. High operator dependency and low reproducibility make ultrasound imaging and interpretation challenging with a steep learning curve. Automatic image classification using deep learning has the potential to overcome some of these challenges by supporting ultrasound training in novices, as well as aiding ultrasound image interpretation in patient with complex pathology for more experienced practitioners. However, the use of deep learning methods requires a large amount of data in order to provide accurate results. Labelling large ultrasound datasets is a challenging task because labels are retrospectively assigned to 2D images without the 3D spatial context available in vivo or that would be inferred while visually tracking structures between frames during the procedure. In this work, we propose a multi-modal convolutional neural network (CNN) architecture that labels endoscopic ultrasound (EUS) images from raw verbal comments provided by a clinician during the procedure. We use a CNN composed of two branches, one for voice data and another for image data, which are joined to predict image labels from the spoken names of anatomical landmarks. The network was trained using recorded verbal comments from expert operators. Our results show a prediction accuracy of 76% at image level on a dataset with 5 different labels. We conclude that the addition of spoken commentaries can increase the performance of ultrasound image classification, and eliminate the burden of manually labelling large EUS datasets necessary for deep learning applications.

Type: Article
Title: Voice-assisted Image Labelling for Endoscopic Ultrasound Classification using Neural Networks
Location: United States
Open access status: An open access version is available from UCL Discovery
DOI: 10.1109/TMI.2021.3139023
Publisher version: https://doi.org/10.1109/TMI.2021.3139023
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Ultrasonic imaging, Training, Labeling, Standards, Real-time systems, Task analysis, Deep learning
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Medical Sciences > Div of Medicine
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Medical Sciences > Div of Medicine > Inst for Liver and Digestive Hlth
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Medical Sciences > Div of Surgery and Interventional Sci
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Medical Sciences > Div of Surgery and Interventional Sci > Department of Surgical Biotechnology
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Med Phys and Biomedical Eng
URI: https://discovery.ucl.ac.uk/id/eprint/10141775
Downloads since deposit
16Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item