Colleoni, E; Moccia, S; Du, X; De Momi, E; Stoyanov, D; (2019) Deep Learning Based Robotic Tool Detection and Articulation Estimation with Spatio-Temporal Layers. IEEE Robotics and Automation Letters , 4 (3) pp. 2714-2721. 10.1109/LRA.2019.2917163.

Preview

Text RA_L_Colleoni_2019.pdf - Accepted Version Download (5MB) | Preview

Abstract

Surgical-tool detection from laparoscopic images is an important but challenging task in computer-assisted minimally invasive surgery. Illumination levels, variations in background and the different number of tools in the field of view, all pose difficulties to algorithm and model training. Yet, such challenges could be potentially tackled by exploiting the temporal information in laparoscopic videos to avoid per frame handling of the problem. In this paper, we propose a novel encoderdecoder architecture for surgical instrument detection and articulation joint detection that uses 3D convolutional layers to exploit spatio-temporal features from laparoscopic videos. When tested on benchmark and custom-built datasets, a median Dice similarity coefficient of 85.1% with an interquartile range of 4.6% highlights performance better than the state of the art based on single-frame processing. Alongside novelty of the network architecture, the idea for inclusion of temporal information appears to be particularly useful when processing images with unseen backgrounds during the training phase, which indicates that spatio-temporal features for joint detection help to generalize the solution.

Download activity - last month

Export as JSONCSVXML

Download activity - last 12 months

Export as JSONCSVXML

Downloads by country - last 12 months

Export as JSONCSVXML

Archive Staff Only

View Item