UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Quantifying the Relationships between Everyday Objects and Emotional States through Deep Learning Based Image Analysis Using Smartphones

Darvariu, V-A; Convertino, L; Mehrotra, A; Musolesi, M; (2020) Quantifying the Relationships between Everyday Objects and Emotional States through Deep Learning Based Image Analysis Using Smartphones. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies , 4 (1) , Article 7. 10.1145/3380997. Green open access

[thumbnail of quantifying-the-relationships-darvariu-et-al-pretypeset.pdf]
Preview
Text
quantifying-the-relationships-darvariu-et-al-pretypeset.pdf - Accepted version

Download (10MB) | Preview

Abstract

There has been an increasing interest in the problem of inferring emotional states of individuals using sensor and user-generated information as diverse as GPS traces, social media data and smartphone interaction patterns. One aspect that has received little attention is the use of visual context information extracted from the surroundings of individuals and how they relate to it. In this paper, we present an observational study of the relationships between the emotional states of individuals and objects present in their visual environment automatically extracted from smartphone images using deep learning techniques. We developed MyMood, a smartphone application that allows users to periodically log their emotional state together with pictures from their everyday lives, while passively gathering sensor measurements. We conducted an in-the-wild study with 22 participants and collected 3,305 mood reports with photos. Our findings show context-dependent associations between objects surrounding individuals and self-reported emotional state intensities. The applications of this work are potentially many, from the design of interior and outdoor spaces to the development of intelligent applications for positive behavioral intervention, and more generally for supporting computational psychology studies.

Type: Article
Title: Quantifying the Relationships between Everyday Objects and Emotional States through Deep Learning Based Image Analysis Using Smartphones
Open access status: An open access version is available from UCL Discovery
DOI: 10.1145/3380997
Publisher version: https://doi.org/10.1145/3380997
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher's terms and conditions.
Keywords: Mobile Sensing; Deep Learning; Digital Mental Health
UCL classification: UCL
UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10096392
Downloads since deposit
173Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item