Birlo, Manuel; (2025) Mixed Reality and Egocentric AI-Assisted Visualisation in Obstetric Ultrasound. Doctoral thesis (Ph.D), UCL (University College London).

Preview

Text Birlo_10216778_Thesis.pdf Download (24MB) | Preview

Abstract

Augmented Reality (AR), particularly in the form of Mixed Reality (MR), holds strong potential to enhance clinical and surgical workflows. However, despite its promise and increasing research interest, demonstrated clinical utility of MR is rare. This thesis investigates specific factors that could contribute to more effective and intuitive AR systems in the future, with the overall goal of supporting real-time AR-assisted decision-making and procedural skill development. First, a systematic literature review of 91 peer-reviewed publications on OST-HMD-based surgical applications was conducted. The review revealed that key technical limitations and human factors remain major barriers to widespread clinical adoption and proposed strategic research directions for improved study design and evaluation. Based on these findings, the focus turned to the underexplored area of AR-assisted training in obstetric ultrasound (US). A novel MR training application (CAL-Tutor) was developed to provide real-time 3D virtual overlay guidance for US probe positioning toward standardised fetal target US planes. A pilot user study demonstrated improved probe navigation and spatial understanding for novel users. Findings from CAL-Tutor led to the development of a new method for egocentric, markerless 3D hand and tool pose estimation. This approach leveraged a scalable synthetic data generation pipeline, combining a generative deep learning model for grasp synthesis with plausible 3D computer graphics-based rendering. The resulting multi-view, multi-modal dataset, HUP-3D, is the first of its kind in the research community. It includes over 31,000 synthetic samples and achieved state-of-the-art single-modality (RGB) accuracy (8.65 mm MPJPE) using the HOPE-Net pose estimation model. A dataset extension (HUP-3D-v2) and initial multi-modal (RGB-D) evaluation finalise the main technical contributions of the thesis. Overall, this thesis contributes to the design of reproducible, scalable and data-driven MR applications for clinical education and interactive procedural training, offering insights for future research at the intersection of MR, egocentric computer vision, and generative deep learning. The findings aim to pave the way for broader adoption of immersive AR- and AI-driven technologies in clinical education and procedural guidance.

Download activity - last month

Export as CSVJSONXML

Download activity - last 12 months

Export as CSVXMLJSON

Downloads by country - last 12 months

Export as XMLJSONCSV

Archive Staff Only

View Item