Li, Liang;
Mazomenos, Evangelos;
Chandler, James H;
Obstein, Keith L;
Valdastri, Pietro;
Stoyanov, Danail;
Vasconcelos, Francisco;
(2023)
Robust endoscopic image mosaicking via fusion of multimodal estimation.
Medical Image Analysis
, 84
, Article 102709. 10.1016/j.media.2022.102709.
Preview |
PDF
1-s2.0-S1361841522003371-main.pdf - Published Version Download (4MB) | Preview |
Abstract
We propose an endoscopic image mosaicking algorithm that is robust to light conditioning changes, specular reflections, and feature-less scenes. These conditions are especially common in minimally invasive surgery where the light source moves with the camera to dynamically illuminate close range scenes. This makes it difficult for a single image registration method to robustly track camera motion and then generate consistent mosaics of the expanded surgical scene across different and heterogeneous environments. Instead of relying on one specialised feature extractor or image registration method, we propose to fuse different image registration algorithms according to their uncertainties, formulating the problem as affine pose graph optimisation. This allows to combine landmarks, dense intensity registration, and learning-based approaches in a single framework. To demonstrate our application we consider deep learning-based optical flow, hand-crafted features, and intensity-based registration, however, the framework is general and could take as input other sources of motion estimation, including other sensor modalities. We validate the performance of our approach on three datasets with very different characteristics to highlighting its generalisability, demonstrating the advantages of our proposed fusion framework. While each individual registration algorithm eventually fails drastically on certain surgical scenes, the fusion approach flexibly determines which algorithms to use and in which proportion to more robustly obtain consistent mosaics.
| Type: | Article |
|---|---|
| Title: | Robust endoscopic image mosaicking via fusion of multimodal estimation |
| Location: | Netherlands |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1016/j.media.2022.102709 |
| Publisher version: | https://doi.org/10.1016/j.media.2022.102709 |
| Language: | English |
| Additional information: | © 2022 The Authors. Published by Elsevier B.V. Under a Creative Commons license (https://creativecommons.org/licenses/by/4.0/). |
| Keywords: | Endoscopic image mosaicking, Image mosaicking, Medical image processing, Optical flow, Pose graph optimisation |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Med Phys and Biomedical Eng |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10162360 |
Archive Staff Only
 |
View Item |
