UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Multimodal Data Fusion based on the Global Workspace Theory

Bao, C; Fountas, Z; Olugbade, T; Bianchi-Berthouze, N; (2020) Multimodal Data Fusion based on the Global Workspace Theory. In: ICMI '20: Proceedings of the 2020 International Conference on Multimodal Interaction. (pp. pp. 414-422). ACM: New York, NY, United States. Green open access

[thumbnail of ICMI_2020 (2).pdf]
Preview
Text
ICMI_2020 (2).pdf - Accepted Version

Download (1MB) | Preview

Abstract

We propose a novel neural network architecture, named the Global Workspace Network (GWN), which addresses the challenge of dynamic and unspecified uncertainties in multimodal data fusion. Our GWN is a model of attention across modalities and evolving through time, and is inspired by the well-established Global Workspace Theory from the field of cognitive science. The GWN achieved average F1 score of 0.92 for discrimination between pain patients and healthy participants and average F1 score = 0.75 for further classification of three pain levels for a patient, both based on the multimodal EmoPain dataset captured from people with chronic pain and healthy people performing different types of exercise movements in unconstrained settings. In these tasks, the GWN significantly outperforms the typical fusion approach of merging by concatenation. We further provide extensive analysis of the behaviour of the GWN and its ability to address uncertainties (hidden noise) in multimodal data.

Type: Proceedings paper
Title: Multimodal Data Fusion based on the Global Workspace Theory
Event: International Conference on Multimodal Interaction
Dates: 25 October 2020 - 29 October 2020
Open access status: An open access version is available from UCL Discovery
DOI: 10.1145/3382507.3418849
Publisher version: https://doi.org/10.1145/3382507.3418849
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences > UCL Interaction Centre
URI: https://discovery.ucl.ac.uk/id/eprint/10109144
Downloads since deposit
141Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item