UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Exploring Multimodal Fusion for Continuous Protective Behavior Detection

Chen, G; Wang, C; Olugbade, T; Williams, A; Berthouze, Nadia; (2022) Exploring Multimodal Fusion for Continuous Protective Behavior Detection. In: Proceedings of the 10th International Conference on Affective Computing and Intelligent Interaction (ACII 2022). (pp. pp. 1-8). Institute of Electrical and Electronics Engineers (IEEE) Green open access

[thumbnail of Exploring_Multimodal_Fusion_for_Protective_Behavior_Detection_in_Continuous_Data_ACII2022-2-final.pdf]
Preview
Text
Exploring_Multimodal_Fusion_for_Protective_Behavior_Detection_in_Continuous_Data_ACII2022-2-final.pdf - Accepted Version

Download (953kB) | Preview

Abstract

Chronic pain is a prevalent condition that affects everyday life of people around the world. Protective behaviors (strategies that are naturally but unhelpfully adopted by people with chronic pain to cope with fear of pain in executing harmless everyday movements) can lead to further disability over time if not recognized and addressed appropriately. In this paper, we build on previous work on unimodal, activity-independent, time-continuous protective behavior detection (PBD) by focusing on the fusion of muscle activity and body movement modalities for characterizing both protective behavior and its physical activity context. We explore different fusion strategies based on consideration of the manner in which protective behavior influences muscle activity and overt body movement as well as the relationship between the two modalities. We evaluate the various strategies on the multimodal EmoPain dataset containing data from people with and without chronic pain engaged in physical activities that reflect everyday challenges for those with chronic pain. Our results show that a central (model-level) fusion approach leads to better PBD performance than input- and decision-level fusions, or unimodal approaches. We also show that additional use of attention mechanism, typifying shifts in attention characteristic of protective behavior, further improves the sensitivity of the model, i.e. detection of the positive class (which is the minority class). We analyze these results and suggest that fusion in modelling a motor condition should consider how emotional responses (fear of movement and pain in this case) triggered by a condition affect each of the given modalities and hence their contributions to the modelling task.

Type: Proceedings paper
Title: Exploring Multimodal Fusion for Continuous Protective Behavior Detection
Event: 10th International Conference on Affective Computing and Intelligent Interaction (ACII) 2022
Location: Nara, Japan
Dates: 18th-22nd October 2022
Open access status: An open access version is available from UCL Discovery
DOI: 10.1109/ACII55700.2022.9953851
Publisher version: https://doi.org/10.1109/ACII55700.2022.9953851
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: chronic pain, deep learning, multimodal fusion
UCL classification: UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences > UCL Interaction Centre
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences
URI: https://discovery.ucl.ac.uk/id/eprint/10153157
Downloads since deposit
5Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item