Joshi, Jitesh Narendra; (2025) Enhancing Out-of-distribution Generalisation for Robust Camera-based Remote Physiological Sensing. Doctoral thesis (Ph.D), UCL (University College London).

Preview

Text Joshi_PhD-Thesis.pdf - Accepted Version Download (100MB) | Preview

Abstract

Colour and thermal infrared imaging enable contactless extraction of physiological signals such as photoplethysmography (PPG) and respiration (RSP), enabling novel applications in human-computer interaction (HCI) and healthcare research. This thesis aims to develop a robust end-to-end learning-based framework for estimating remote photoplethysmography (rPPG) and remote respiration (rRSP) signals with enhanced out-of-distribution generalisation for real-world deployment. To reach this aim, this work makes a set of contributions. First, an open-source physiological computing toolkit, PhysioKit, was developed to acquire iBVP dataset that comprise synchronised RGB and thermal (RGB-T) video with multichannel physiological signals from contact-based sensors, along with real-time signal quality assessment. Second, this thesis introduces a Self-Adversarial Multiscale Contrastive Learning Framework (SAM-CL) for robust segmentation of thermal facial frames under adverse conditions and occlusions. SAM-CL has a thermal image augmentation module (TiAug), that generates adversarial samples and segmentation masks, and a multiscale contrastive loss that optimises the segmentation model. Third, to address out-of-distribution generalisation for physiological signal extraction, this work proposes FactorisePhys, a lightweight 3D-CNN architecture, and the Factorised SelfAttention Module (FSAM), mathematically grounded in non-negative matrix factorisation (NMF). FSAM selectively boosts weaker physiological signals in video frames while remaining invariant to lighting and head movements. FactorisePhys, trained with FSAM, shows superior cross-dataset generalisation for rPPG signal estimation as compared to the existing state-of-the-art (SOTA) methods. Finally, building on FSAM and advances in constrained NMF, the Target Signal Constrained Factorisation Module (TSFM) and MMRPhys, an efficient 3D-CNN architecture, are proposed for robust multitask rPPG and rRSP signal estimation using RGB-T frames. MMRPhys, trained with TSFM, shows superior cross-dataset generalisation as compared to the SOTA multitask models along with achieving the lowest inference time latency, making it suitable for real-time edge deployment. In summary, this thesis makes notable contributions to enhance out-of-distribution robustness in remote physiological sensing research.

Download activity - last month

Export as JSONCSVXML

Download activity - last 12 months

Export as JSONCSVXML

Downloads by country - last 12 months

Export as XMLCSVJSON

Archive Staff Only

View Item