UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Instant Stress: Detection of Perceived Mental Stress Through Smartphone Photoplethysmography and Thermal Imaging

Cho, Y; Julier, SJ; Bianchi-Berthouze, N; (2019) Instant Stress: Detection of Perceived Mental Stress Through Smartphone Photoplethysmography and Thermal Imaging. JMIR Mental Health , 6 (4) , Article e10140. 10.2196/10140. Green open access

[img]
Preview
Text
Cho_c53e3180-0fda-4ea5-9c09-2b11ff37785e.pdf - Published version

Download (3MB) | Preview

Abstract

Background: A smartphone is a promising tool for daily cardiovascular measurement and mental stress monitoring. A smartphone camera-based PhotoPlethysmoGraphy (PPG) and a low-cost thermal camera can be used to create cheap, convenient and mobile monitoring systems. However, to ensure reliable monitoring results, a person has to remain still for several minutes while a measurement is being taken. This is very cumbersome and makes its use in real-life mobile situations quite impractical. // Objective: We propose a system which combines PPG and thermography with the aim of improving cardiovascular signal quality and capturing stress responses quickly. // Methods: Using a smartphone camera with a low cost thermal camera added on, we built a novel system which continuously and reliably measures two different types of cardiovascular events: i) blood volume pulse and ii) vasoconstriction/dilation-induced temperature changes of the nose tip. 17 healthy participants, involved in a series of stress-inducing mental workload tasks, measured their physiological responses to stressors over a short window of time (20 seconds) immediately after each task. Participants reported their level of perceived mental stress using a 10-cm Visual Analogue Scale (VAS). We used normalized K-means clustering to reduce interpersonal differences in the self-reported ratings. For the instant stress inference task, we built novel low-level feature sets representing variability of cardiovascular patterns. We then used the automatic feature learning capability of artificial Neural Networks (NN) to improve the mapping between the extracted set of features and the self-reported ratings. We compared our proposed method with existing hand-engineered features-based machine learning methods. // Results: First, we found that the measured PPG signals presented high quality cardiac cyclic information (relative power Signal Quality Index, pSQI: M=0.755, SD=0.068). We also found that the measured thermal changes of the nose tip presented high quality breathing cyclic information and filtering helped extract vasoconstriction/dilation-induced patterns with fewer respiratory effects (respiratory pSQI: from M=0.714 to M=0.157). Second, we found low correlations between the self-reported stress scores and the existing metrics of the two cardiovascular signals (i.e. heart rate variability and thermal directionality metrics) from short measurements, suggesting they were not very dependent upon one another. Third, we tested the performance of the instant perceived stress inference method. The proposed method achieved significantly higher accuracies than existing pre-crafted features based-methods. In addition, the 17-fold Leave-One-Subject-Out (LOSO) cross-validation results showed that combination of both modalities produced higher accuracy in comparison with the use of PPG or thermal imaging only (PPG+Thermal: 78.33%; PPG: 68.53%; Thermal: 58.82%). The multimodal results are comparable to the state-of-the-art automatic stress recognition methods that require long term measurements (usually, at least a period of 2 minutes is required for an accuracy of around 80% from LOSO). Lastly, we explored effects of different widely-used data labeling strategies on the sensitivity of our inference methods. Our results showed the need for separation of and normalization between individual data. // Conclusions: Results demonstrate the feasibility of using smartphone-based imaging for instant mental stress recognition. Given that this approach does not need long-term measurements requiring attention and reduced mobility, we believe it is more suitable for mobile mental healthcare solutions in the wild.

Type: Article
Title: Instant Stress: Detection of Perceived Mental Stress Through Smartphone Photoplethysmography and Thermal Imaging
Open access status: An open access version is available from UCL Discovery
DOI: 10.2196/10140
Publisher version: https://doi.org/10.2196/10140
Language: English
Additional information: This work is licensed under the Creative Commons Attribution License CC-BY 2.0 (http://www.creativecommons.org/licenses/by/2.0/).
Keywords: Automatic stress recognition, mobile devices, PPG, low-cost thermal imaging, blood volume pulse, variability, nose temperature, physiological computing, affective computing, machine learning
UCL classification: UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences > UCL Interaction Centre
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10064710
Downloads since deposit
33Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item