UCL logo

UCL Discovery

UCL home » Library Services » Electronic resources » UCL Discovery

Photorealistic retrieval of occluded facial information using a performance-driven face model

Berisha, F.; (2009) Photorealistic retrieval of occluded facial information using a performance-driven face model. Doctoral thesis, UCL (University College London). Green open access

[img]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
1898Kb
[img]AVI video (Supplementary video from Chapter 3: First 15 PCs extracted from a natural facial motion sequence, shown here oscillating +/- 2 standard deviations from the mean)
41Mb
[img]AVI video (Supplementary video from Chapter 3: Example demonstrating PCA-based performance driven mimicry, transferring Jason's behaviour onto Joanne's face)
6Mb
[img]AVI video (Supplementary video from Chapter 3: Example demonstrating PCA-based performance driven mimicry, transferring Joanne's behaviour onto Jason's face)
8Mb
[img]AVI video (Supplementary video from Chapter 4: Example of face-map mimicry, Glyn to Fatos)
1859Kb
[img]AVI video (Supplementary video from Chapter 4: Example of "good" bubblemask mimicry, as categorised by the PC coefficient comparison with ground-truth)
1859Kb
[img]AVI video (Supplementary video from Chapter 4: Example of "bad" bubblemask mimicry, as categorised by the PC coefficient comparison with ground-truth)
1859Kb
[img]AVI video (Supplementary video from Chapter 5: Mouth occlusion affected mimicry, no PC exaggeration)
46Mb
[img]AVI video (Supplementary video from Chapter 5: Mouth occlusion affected mimicry, PC coefficients multiplied by 1.5)
46Mb
[img]AVI video (AVI video (Supplementary video from Chapter 5: Mouth occlusion affected mimicry, PC coefficients multiplied by 2)
46Mb
[img]AVI video (Supplementary video from Chapter 5: Mouth occlusion affected mimicry, PC coefficients multiplied by 3)
46Mb
[img]AVI video (Supplementary video from Chapter 5: Mouth occlusion affected mimicry, PC coefficients multiplied by 4.5)
46Mb
[img]AVI video (Supplementary video from Chapter 6: First 30 PCs extracted from a natural facial motion sequence (Glyn), shown here oscillating +/- 2 standard deviations from the mean)
20Mb
[img]AVI video (Supplementary video from Chapter 6: First 33 ICs extracted from a natural facial motion sequence (Glyn), shown here oscillating +/- 2 standard deviations from the mean)
22Mb
[img]AVI video (Supplementary video from Chapter 6: First 33 ICs extracted from a natural facial motion sequence (Fatos), shown here oscillating +/- 2 standard deviations from the mean)
22Mb
[img]AVI video (Supplementary video from Chapter 6: Example of ICA-enhanced mimicry (Glyn to Fatos). Left-most frame is occluded driver, followed by plain PCA mimicry, ICA-enhanced mimicry and ground-truth)
3715Kb
[img]AVI video (Supplementary video from Chapter 6: Example of ICA-enhanced mimicry (Alison to Glyn). Left-most frame is occluded driver, followed by plain PCA mimicry, ICA-enhanced mimicry and ground-truth)
3715Kb

Abstract

Facial occlusions can cause both human observers and computer algorithms to fail in a variety of important tasks such as facial action analysis and expression classification. This is because the missing information is not reconstructed accurately enough for the purpose of the task in hand. Most current computer methods that are used to tackle this problem implement complex three-dimensional polygonal face models that are generally timeconsuming to produce and unsuitable for photorealistic reconstruction of missing facial features and behaviour. In this thesis, an image-based approach is adopted to solve the occlusion problem. A dynamic computer model of the face is used to retrieve the occluded facial information from the driver faces. The model consists of a set of orthogonal basis actions obtained by application of principal component analysis (PCA) on image changes and motion fields extracted from a sequence of natural facial motion (Cowe 2003). Examples of occlusion affected facial behaviour can then be projected onto the model to compute coefficients of the basis actions and thus produce photorealistic performance-driven animations. Visual inspection shows that the PCA face model recovers aspects of expressions in those areas occluded in the driver sequence, but the expression is generally muted. To further investigate this finding, a database of test sequences affected by a considerable set of artificial and natural occlusions is created. A number of suitable metrics is developed to measure the accuracy of the reconstructions. Regions of the face that are most important for performance-driven mimicry and that seem to carry the best information about global facial configurations are revealed using Bubbles, thus in effect identifying facial areas that are most sensitive to occlusions. Recovery of occluded facial information is enhanced by applying an appropriate scaling factor to the respective coefficients of the basis actions obtained by PCA. This method improves the reconstruction of the facial actions emanating from the occluded areas of the face. However, due to the fact that PCA produces bases that encode composite, correlated actions, such an enhancement also tends to affect actions in non-occluded areas of the face. To avoid this, more localised controls for facial actions are produced using independent component analysis (ICA). Simple projection of the data onto an ICA model is not viable due to the non-orthogonality of the extracted bases. Thus occlusion-affected mimicry is first generated using the PCA model and then enhanced by accordingly manipulating the independent components that are subsequently extracted from the mimicry. This combination of methods yields significant improvements and results in photorealistic reconstructions of occluded facial actions.

Type:Thesis (Doctoral)
Title:Photorealistic retrieval of occluded facial information using a performance-driven face model
Open access status:An open access version is available from UCL Discovery
Language:English
UCL classification:UCL > School of BEAMS > Faculty of Maths and Physical Sciences > CoMPLEX - Maths and Physics in the Life Sciences and Experimental Biology
UCL > School of Life and Medical Sciences > Faculty of Brain Sciences > Psychology and Language Sciences (Division of) > Clinical, Educational and Health Psychology

View download statistics for this item

Archive Staff Only: edit this record