UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Automatic generation of operation notes in endoscopic pituitary surgery videos using workflow recognition

Das, A; Khan, DZ; Hanrahan, JG; Marcus, HJ; Stoyanov, D; (2023) Automatic generation of operation notes in endoscopic pituitary surgery videos using workflow recognition. Intelligence-Based Medicine , 8 , Article 100107. 10.1016/j.ibmed.2023.100107. Green open access

[thumbnail of Das_Automatic generation of operation notes in endoscopic pituitary surgery videos using workflow recognition_VoR.pdf]
Preview
Text
Das_Automatic generation of operation notes in endoscopic pituitary surgery videos using workflow recognition_VoR.pdf - Published Version

Download (4MB) | Preview

Abstract

Operation notes are a crucial component of patient care. However, writing them manually is prone to human error, particularly in high pressured clinical environments. Automatic generation of operation notes from video recordings can alleviate some of the administrative burdens, improve accuracy, and provide additional information. To achieve this for endoscopic pituitary surgery, 27-steps were identified via expert consensus. Then, for the 97-videos recorded for this study, a timestamp of each step was annotated by an expert surgeon. To automatically determine whether a step is present in a video, a three-stage architecture was created. Firstly, for each step, a convolution neural network was used for binary image classification on each frame of a video. Secondly, for each step, the binary frame classifications were passed to a discriminator for binary video classification. Thirdly, for each video, the binary video classifications were passed to an accumulator for multi-label step classification. The architecture was trained on 77-videos, and tested on 20-videos, where a 0.80 weighted-F1 score was achieved. The classifications were inputted into a clinically based predefined template, and further enriched with additional video analytics. This work therefore demonstrates automatic generation of operative notes from surgical videos is feasible, and can assist surgeons during documentation.

Type: Article
Title: Automatic generation of operation notes in endoscopic pituitary surgery videos using workflow recognition
Open access status: An open access version is available from UCL Discovery
DOI: 10.1016/j.ibmed.2023.100107
Publisher version: https://doi.org/10.1016/j.ibmed.2023.100107
Language: English
Additional information: © 2023 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
Keywords: Computer vision, Image recognition, Operation report, Step recognition, Surgical AI, Workflow analysis
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Health Informatics
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology > Department of Neuromuscular Diseases
URI: https://discovery.ucl.ac.uk/id/eprint/10178640
Downloads since deposit
15Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item