eprintid: 10124990 rev_number: 16 eprint_status: archive userid: 608 dir: disk0/10/12/49/90 datestamp: 2021-03-26 10:31:19 lastmod: 2021-09-23 22:25:19 status_changed: 2021-03-26 10:31:19 type: article metadata_visibility: show creators_name: Szczotka, AB creators_name: Shakir, DI creators_name: Clarkson, MJ creators_name: Pereira, SP creators_name: Vercauteren, T title: Zero-shot super-resolution with a physically-motivated downsampling kernel for endomicroscopy ispublished: inpress divisions: UCL divisions: B02 divisions: C10 divisions: D17 divisions: G91 divisions: B04 divisions: C05 divisions: F42 keywords: blind super-resolution, endomicroscopy, video enhancement, zero-shot note: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. abstract: Super-resolution (SR) methods have seen significant advances thanks to the development of convolutional neural networks (CNNs). CNNs have been successfully employed to improve the quality of endomicroscopy imaging. Yet, the inherent limitation of research on SR in endomicroscopy remains the lack of ground truth high-resolution (HR) images, commonly used for both supervised training and reference-based image quality assessment (IQA). Therefore, alternative methods, such as unsupervised SR are being explored. To address the need for non-reference image quality improvement, we designed a novel zero-shot super-resolution (ZSSR) approach that relies only on the endomicroscopy data to be processed in a self-supervised manner without the need for ground-truth HR images. We tailored the proposed pipeline to the idiosyncrasies of endomicroscopy by introducing both: a physically-motivated Voronoi downscaling kernel accounting for the endomicroscope’s irregular fibre-based sampling pattern, and realistic noise patterns. We also took advantage of video sequences to exploit a sequence of images for self-supervised zero-shot image quality improvement. We run ablation studies to assess our contribution in regards to the downscaling kernel and noise simulation. We validate our methodology on both synthetic and original data. Synthetic experiments were assessed with reference-based IQA, while our results for original images were evaluated in a user study conducted with both expert and non-expert observers. The results demonstrated superior performance in image quality of ZSSR reconstructions in comparison to the baseline method. The ZSSR is also competitive when compared to supervised single-image SR, especially being the preferred reconstruction technique by experts. date: 2021-03-19 date_type: published official_url: https://dx.doi.org/10.1109/TMI.2021.3067512 oa_status: green full_text_type: other language: eng primo: open primo_central: open_green verified: verified_manual elements_id: 1855111 doi: 10.1109/TMI.2021.3067512 lyricists_name: Clarkson, Matthew lyricists_name: Pereira, Stephen lyricists_name: Szczotka, Agnieszka lyricists_name: Vercauteren, Tom lyricists_id: MJCLA42 lyricists_id: SPPER57 lyricists_id: ABSZC96 lyricists_id: TVERC65 actors_name: Clarkson, Matthew actors_id: MJCLA42 actors_role: owner full_text_status: public publication: IEEE Transactions on Medical Imaging event_location: United States issn: 1558-254X citation: Szczotka, AB; Shakir, DI; Clarkson, MJ; Pereira, SP; Vercauteren, T; (2021) Zero-shot super-resolution with a physically-motivated downsampling kernel for endomicroscopy. IEEE Transactions on Medical Imaging 10.1109/TMI.2021.3067512 <https://doi.org/10.1109/TMI.2021.3067512>. (In press). Green open access document_url: https://discovery.ucl.ac.uk/id/eprint/10124990/1/2021SzczotkaZSSR_Accepted.pdf