Psychogyios, D; Vasconcelos, F; Stoyanov, D; (2023) Realistic Endoscopic Illumination Modeling for NeRF-Based Data Generation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention MICCAI 2023: Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. (pp. pp. 535-544). Springer, Cham

Text miccai_2023_nerf_final.pdf - Accepted Version Access restricted to UCL open access staff until 2 October 2024. Download (4MB)

Abstract

Expanding training and evaluation data is a major step towards building and deploying reliable localization and 3D reconstruction techniques during colonoscopy screenings. However, training and evaluating pose and depth models in colonoscopy is hard as available datasets are limited in size. This paper proposes a method for generating new pose and depth datasets by fitting NeRFs in already available colonoscopy datasets. Given a set of images, their associated depth maps and pose information, we train a novel light source location-conditioned NeRF to encapsulate the 3D and color information of a colon sequence. Then, we leverage the trained networks to render images from previously unobserved camera poses and simulate different camera systems, effectively expanding the source dataset. Our experiments show that our model is able to generate RGB images and depth maps of a colonoscopy sequence from previously unobserved poses with high accuracy. Code and trained networks can be accessed at

Download activity - last month

Export as XMLCSVJSON

Download activity - last 12 months

Export as CSVXMLJSON

Downloads by country - last 12 months

Export as JSONCSVXML

Archive Staff Only

View Item