UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Image Compositing for Segmentation of Surgical Tools Without Manual Annotations

García-Peraza-Herrera, LC; Fidon, L; D'Ettorre, C; Stoyanov, D; Vercauteren, T; Ourselin, S; (2021) Image Compositing for Segmentation of Surgical Tools Without Manual Annotations. IEEE Transactions on Medical Imaging , 40 (5) pp. 1450-1460. 10.1109/TMI.2021.3057884. Green open access

[thumbnail of 09350303.pdf]
Preview
Text
09350303.pdf - Published Version

Download (1MB) | Preview

Abstract

Producing manual, pixel-accurate, image segmentation labels is tedious and time-consuming. This is often a rate-limiting factor when large amounts of labeled images are required, such as for training deep convolutional networks for instrument-background segmentation in surgical scenes. No large datasets comparable to industry standards in the computer vision community are available for this task. To circumvent this problem, we propose to automate the creation of a realistic training dataset by exploiting techniques stemming from special effects and harnessing them to target training performance rather than visual appeal. Foreground data is captured by placing sample surgical instruments over a chroma key (a.k.a. green screen) in a controlled environment, thereby making extraction of the relevant image segment straightforward. Multiple lighting conditions and viewpoints can be captured and introduced in the simulation by moving the instruments and camera and modulating the light source. Background data is captured by collecting videos that do not contain instruments. In the absence of pre-existing instrument-free background videos, minimal labeling effort is required, just to select frames that do not contain surgical instruments from videos of surgical interventions freely available online. We compare different methods to blend instruments over tissue and propose a novel data augmentation approach that takes advantage of the plurality of options. We show that by training a vanilla U-Net on semi-synthetic data only and applying a simple post-processing, we are able to match the results of the same network trained on a publicly available manually labeled real dataset.

Type: Article
Title: Image Compositing for Segmentation of Surgical Tools Without Manual Annotations
Open access status: An open access version is available from UCL Discovery
DOI: 10.1109/TMI.2021.3057884
Publisher version: http://dx.doi.org/10.1109/TMI.2021.3057884
Language: English
Additional information: This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
Keywords: Image segmentation , Instruments, Tools, Training, Task analysis, Surgery, Manuals
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Med Phys and Biomedical Eng
URI: https://discovery.ucl.ac.uk/id/eprint/10128684
Downloads since deposit
70Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item