UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Self-knowledge distillation for surgical phase recognition

Zhang, J; Barbarisi, S; Kadkhodamohammadi, A; Stoyanov, D; Luengo, I; (2024) Self-knowledge distillation for surgical phase recognition. International Journal of Computer Assisted Radiology and Surgery , 19 pp. 61-68. 10.1007/s11548-023-02970-7. Green open access

[thumbnail of Self-knowledge distillation for surgical phase recognition.pdf]
Preview
Text
Self-knowledge distillation for surgical phase recognition.pdf - Accepted Version

Download (579kB) | Preview

Abstract

Purpose: Advances in surgical phase recognition are generally led by training deeper networks. Rather than going further with a more complex solution, we believe that current models can be exploited better. We propose a self-knowledge distillation framework that can be integrated into current state-of-the-art (SOTA) models without requiring any extra complexity to the models or annotations. Methods: Knowledge distillation is a framework for network regularization where knowledge is distilled from a teacher network to a student network. In self-knowledge distillation, the student model becomes the teacher such that the network learns from itself. Most phase recognition models follow an encoder-decoder framework. Our framework utilizes self-knowledge distillation in both stages. The teacher model guides the training process of the student model to extract enhanced feature representations from the encoder and build a more robust temporal decoder to tackle the over-segmentation problem. Results: We validate our proposed framework on the public dataset Cholec80. Our framework is embedded on top of four popular SOTA approaches and consistently improves their performance. Specifically, our best GRU model boosts performance by + 3.33% accuracy and + 3.95% F1-score over the same baseline model. Conclusion: We embed a self-knowledge distillation framework for the first time in the surgical phase recognition training pipeline. Experimental results demonstrate that our simple yet powerful framework can improve performance of existing phase recognition models. Moreover, our extensive experiments show that even with 75% of the training set we still achieve performance on par with the same baseline model trained on the full set.

Type: Article
Title: Self-knowledge distillation for surgical phase recognition
Location: Germany
Open access status: An open access version is available from UCL Discovery
DOI: 10.1007/s11548-023-02970-7
Publisher version: https://doi.org/10.1007/s11548-023-02970-7
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Surgical phase recognition, Knowledge distillation, Self-supervised learning, Surgical data science
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10175473
Downloads since deposit
4Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item