UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Deep Disturbance-disentangled Learning for Facial Expression Recognition

Ruan, D; Yan, Y; Chen, S; Xue, J; Wang, H; (2020) Deep Disturbance-disentangled Learning for Facial Expression Recognition. In: MM '20: Proceedings of the 28th ACM International Conference on Multimedia. (pp. pp. 2833-2841). ACM Green open access

[thumbnail of DelianRuan-ACMMM2020-accepted.pdf] Text
DelianRuan-ACMMM2020-accepted.pdf - Accepted Version

Download (2MB)

Abstract

To achieve effective facial expression recognition (FER), it is of great importance to address various disturbing factors, including pose, illumination, identity, and so on. However, a number of FER databases merely provide the labels of facial expression, identity, and pose, but lack the label information for other disturbing factors. As a result, many methods are only able to cope with one or two disturbing factors, ignoring the heavy entanglement between facial expression and multiple disturbing factors. In this paper, we propose a novel Deep Disturbance-disentangled Learning (DDL) method for FER. DDL is capable of simultaneously and explicitly disentangling multiple disturbing factors by taking advantage of multi-task learning and adversarial transfer learning. The training of DDL involves two stages. First, a Disturbance Feature Extraction Model (DFEM) is pre-trained to perform multi-task learning for classifying multiple disturbing factors on the large-scale face database (which has the label information for various disturbing factors). Second, a Disturbance-Disentangled Model (DDM), which contains a global shared sub-network and two task-specific (i.e., expression and disturbance) sub-networks, is learned to encode the disturbance-disentangled information for expression recognition. The expression sub-network adopts a multi-level attention mechanism to extract expression-specific features, while the disturbance sub-network leverages adversarial transfer learning to extract disturbance-specific features based on the pre-trained DFEM. Experimental results on both the in-the-lab FER databases (including CK+, MMI, and Oulu-CASIA) and the in-the-wild FER databases (including RAF-DB and SFEW) demonstrate the superiority of our proposed method compared with several state-of-the-art methods.

Type: Proceedings paper
Title: Deep Disturbance-disentangled Learning for Facial Expression Recognition
Event: ACM Multimedia 2020
Dates: 12 October 2020 - 16 October 2020
Open access status: An open access version is available from UCL Discovery
DOI: 10.1145/3394171.3413907
Publisher version: https://doi.org/10.1145/3394171.3413907
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences > Dept of Statistical Science
URI: https://discovery.ucl.ac.uk/id/eprint/10110888
Downloads since deposit
303Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item