B R, Mallikarjun;
Tewari, Ayush;
Oh, Tae-Hyun;
Weyrich, Tim;
Bickel, Bernd;
Seidel, Hans-Peter;
Pfister, Hanspeter;
... Theobalt, Christian; + view all
(2021)
Monocular Reconstruction of Neural Face Reflectance Fields.
In:
Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
(pp. pp. 4789-4798).
IEEE: Nashville, TN, USA.
Preview |
Text
paper.pdf - Accepted Version Download (21MB) | Preview |
Abstract
The reflectance field of a face describes the reflectance properties responsible for complex lighting effects including diffuse, specular, inter-reflection and self shadowing. Most existing methods for estimating the face reflectance from a monocular image assume faces to be diffuse with very few approaches adding a specular component. This still leaves out important perceptual aspects of reflectance such as higher-order global illumination effects and self-shadowing. We present a new neural representation for face reflectance where we can estimate all components of the reflectance responsible for the final appearance from a monocular image. Instead of modeling each component of the reflectance separately using parametric models, our neural representation allows us to generate a basis set of faces in a geometric deformation-invariant space, parameterized by the input light direction, viewpoint and face geometry. We learn to reconstruct this reflectance field of a face just from a monocular image, which can be used to render the face from any viewpoint in any light condition. Our method is trained on a light-stage dataset, which captures 300 people illuminated with 150 light conditions from 8 viewpoints. We show that our method outperforms existing monocular reflectance reconstruction methods due to better capturing of physical effects, such as sub-surface scattering, specularities, self-shadows and other higher-order effects.
| Type: | Proceedings paper |
|---|---|
| Title: | Monocular Reconstruction of Neural Face Reflectance Fields |
| Event: | 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) |
| Location: | ELECTR NETWORK |
| Dates: | 19 Jun 2021 - 25 Jun 2021 |
| ISBN-13: | 978-1-6654-4509-2 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/CVPR46437.2021.00476 |
| Publisher version: | https://doi.org/10.1109/CVPR46437.2021.00476 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | Reflectivity, Geometry, Deformable models, Face recognition, Lighting, Scattering, Reconstruction algorithms |
| UCL classification: | UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science UCL > Provost and Vice Provost Offices > UCL BEAMS UCL |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10152573 |
Archive Staff Only
 |
View Item |
