UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Time-Delay Neural Network for Continuous Emotional Dimension Prediction From Facial Expression Sequences

Meng, H; Bianchi-Berthouze, N; Deng, Y; Cheng, J; Cosmas, JP; (2016) Time-Delay Neural Network for Continuous Emotional Dimension Prediction From Facial Expression Sequences. IEEE Transactions on Cybernetics , 46 (4) pp. 916-929. 10.1109/TCYB.2015.2418092. Green open access

[thumbnail of TCYB2418092.pdf]
Preview
Text
TCYB2418092.pdf - Published Version

Download (1MB) | Preview

Abstract

Automatic continuous affective state prediction from naturalistic facial expression is a very challenging research topic but very important in human-computer interaction. One of the main challenges is modeling the dynamics that characterize naturalistic expressions. In this paper, a novel two-stage automatic system is proposed to continuously predict affective dimension values from facial expression videos. In the first stage, traditional regression methods are used to classify each individual video frame, while in the second stage, a time-delay neural network (TDNN) is proposed to model the temporal relationships between consecutive predictions. The two-stage approach separates the emotional state dynamics modeling from an individual emotional state prediction step based on input features. In doing so, the temporal information used by the TDNN is not biased by the high variability between features of consecutive frames and allows the network to more easily exploit the slow changing dynamics between emotional states. The system was fully tested and evaluated on three different facial expression video datasets. Our experimental results demonstrate that the use of a two-stage approach combined with the TDNN to take into account previously classified frames significantly improves the overall performance of continuous emotional state estimation in naturalistic facial expressions. The proposed approach has won the affect recognition sub-challenge of the Third International Audio/Visual Emotion Recognition Challenge.

Type: Article
Title: Time-Delay Neural Network for Continuous Emotional Dimension Prediction From Facial Expression Sequences
Open access status: An open access version is available from UCL Discovery
DOI: 10.1109/TCYB.2015.2418092
Publisher version: https://doi.org/10.1109/TCYB.2015.2418092
Language: English
Additional information: Copyright © the Authors 2016. Open Access. This work is licensed under a Creative Commons Attribution 3.0 License. For more information, see http://creativecommons.org/ licenses/by/3.0/.
Keywords: Affective computing, emotion dimension, emotion prediction, facial expression, neural networks
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences > UCL Interaction Centre
URI: https://discovery.ucl.ac.uk/id/eprint/10074442
Downloads since deposit
81Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item