eprintid: 10149144
rev_number: 8
eprint_status: archive
userid: 699
dir: disk0/10/14/91/44
datestamp: 2022-05-26 10:52:18
lastmod: 2023-03-31 06:10:15
status_changed: 2022-05-26 10:52:18
type: article
metadata_visibility: show
sword_depositor: 699
creators_name: Kadkhodamohammadi, Abdolrahim
creators_name: Luengo, Imanol
creators_name: Stoyanov, Danail
title: PATG: position-aware temporal graph networks for surgical phase recognition on laparoscopic videos
ispublished: pub
divisions: C05
divisions: F48
divisions: B04
divisions: UCL
keywords: Graph neural networks, Positional encoder, Surgical AI, Surgical data science, Surgical phase recognition, Workflow analysis
note: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
abstract: PURPOSE: We tackle the problem of online surgical phase recognition in laparoscopic procedures, which is key in developing context-aware supporting systems. We propose a novel approach to take temporal context in surgical videos into account by precise modeling of temporal neighborhoods. METHODS: We propose a two-stage model to perform phase recognition. A CNN model is used as a feature extractor to project RGB frames into a high-dimensional feature space. We introduce a novel paradigm for surgical phase recognition which utilizes graph neural networks to incorporate temporal information. Unlike recurrent neural networks and temporal convolution networks, our graph-based approach offers a more generic and flexible way for modeling temporal relationships. Each frame is a node in the graph, and the edges in the graph are used to define temporal connections among the nodes. The flexible configuration of temporal neighborhood comes at the price of losing temporal order. To mitigate this, our approach takes temporal orders into account by encoding frame positions, which is important to reliably predict surgical phases. RESULTS: Experiments are carried out on the public Cholec80 dataset that contains 80 annotated videos. The experimental results highlight the superior performance of the proposed approach compared to the state-of-the-art models on this dataset. CONCLUSION: A novel approach for formulating video-based surgical phase recognition is presented. The results indicate that temporal information can be incorporated using graph-based models, and positional encoding is important to efficiently utilize temporal information. Graph networks open possibilities to use evidence theory for uncertainty analysis in surgical phase recognition.
date: 2022-05
date_type: published
publisher: Springer Verlag
official_url: https://doi.org/10.1007/s11548-022-02600-8
oa_status: green
full_text_type: other
language: eng
primo: open
primo_central: open_green
verified: verified_manual
elements_id: 1948720
doi: 10.1007/s11548-022-02600-8
medium: Print-Electronic
pii: 10.1007/s11548-022-02600-8
lyricists_name: Stoyanov, Danail
lyricists_id: DSTOY26
actors_name: Stoyanov, Danail
actors_name: Woolley, Clare
actors_id: DSTOY26
actors_id: CWOOL20
actors_role: owner
actors_role: impersonator
full_text_status: public
publication: International Journal of Computer Assisted Radiology and Surgery
volume: 17
number: 5
pagerange: 849-856
pages: 8
event_location: Germany
citation:        Kadkhodamohammadi, Abdolrahim;    Luengo, Imanol;    Stoyanov, Danail;      (2022)    PATG: position-aware temporal graph networks for surgical phase recognition on laparoscopic videos.                   International Journal of Computer Assisted Radiology and Surgery , 17  (5)   pp. 849-856.    10.1007/s11548-022-02600-8 <https://doi.org/10.1007/s11548-022-02600-8>.       Green open access   
 
document_url: https://discovery.ucl.ac.uk/id/eprint/10149144/1/PATG%20%281%29.pdf