eprintid: 10149144 rev_number: 8 eprint_status: archive userid: 699 dir: disk0/10/14/91/44 datestamp: 2022-05-26 10:52:18 lastmod: 2023-03-31 06:10:15 status_changed: 2022-05-26 10:52:18 type: article metadata_visibility: show sword_depositor: 699 creators_name: Kadkhodamohammadi, Abdolrahim creators_name: Luengo, Imanol creators_name: Stoyanov, Danail title: PATG: position-aware temporal graph networks for surgical phase recognition on laparoscopic videos ispublished: pub divisions: C05 divisions: F48 divisions: B04 divisions: UCL keywords: Graph neural networks, Positional encoder, Surgical AI, Surgical data science, Surgical phase recognition, Workflow analysis note: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. abstract: PURPOSE: We tackle the problem of online surgical phase recognition in laparoscopic procedures, which is key in developing context-aware supporting systems. We propose a novel approach to take temporal context in surgical videos into account by precise modeling of temporal neighborhoods. METHODS: We propose a two-stage model to perform phase recognition. A CNN model is used as a feature extractor to project RGB frames into a high-dimensional feature space. We introduce a novel paradigm for surgical phase recognition which utilizes graph neural networks to incorporate temporal information. Unlike recurrent neural networks and temporal convolution networks, our graph-based approach offers a more generic and flexible way for modeling temporal relationships. Each frame is a node in the graph, and the edges in the graph are used to define temporal connections among the nodes. The flexible configuration of temporal neighborhood comes at the price of losing temporal order. To mitigate this, our approach takes temporal orders into account by encoding frame positions, which is important to reliably predict surgical phases. RESULTS: Experiments are carried out on the public Cholec80 dataset that contains 80 annotated videos. The experimental results highlight the superior performance of the proposed approach compared to the state-of-the-art models on this dataset. CONCLUSION: A novel approach for formulating video-based surgical phase recognition is presented. The results indicate that temporal information can be incorporated using graph-based models, and positional encoding is important to efficiently utilize temporal information. Graph networks open possibilities to use evidence theory for uncertainty analysis in surgical phase recognition. date: 2022-05 date_type: published publisher: Springer Verlag official_url: https://doi.org/10.1007/s11548-022-02600-8 oa_status: green full_text_type: other language: eng primo: open primo_central: open_green verified: verified_manual elements_id: 1948720 doi: 10.1007/s11548-022-02600-8 medium: Print-Electronic pii: 10.1007/s11548-022-02600-8 lyricists_name: Stoyanov, Danail lyricists_id: DSTOY26 actors_name: Stoyanov, Danail actors_name: Woolley, Clare actors_id: DSTOY26 actors_id: CWOOL20 actors_role: owner actors_role: impersonator full_text_status: public publication: International Journal of Computer Assisted Radiology and Surgery volume: 17 number: 5 pagerange: 849-856 pages: 8 event_location: Germany citation: Kadkhodamohammadi, Abdolrahim; Luengo, Imanol; Stoyanov, Danail; (2022) PATG: position-aware temporal graph networks for surgical phase recognition on laparoscopic videos. International Journal of Computer Assisted Radiology and Surgery , 17 (5) pp. 849-856. 10.1007/s11548-022-02600-8 <https://doi.org/10.1007/s11548-022-02600-8>. Green open access document_url: https://discovery.ucl.ac.uk/id/eprint/10149144/1/PATG%20%281%29.pdf