Giebenhain, Simon;
Kirschstein, Tobias;
Rünz, Martin;
Agapito, Lourdes;
Nießner, Matthias;
(2024)
NPGA: Neural Parametric Gaussian Avatars.
In: Igarashi, Takeo and Shamir, Ariel and Zhang, Hao (Richard), (eds.)
SA '24: SIGGRAPH Asia 2024 Conference Papers.
(pp. Article No-127).
ACM (Association for Computing Machinery): New York, NY, United States.
Preview |
Text
De Agapito Vicente_NPGA_3680528.3687689.pdf Download (13MB) | Preview |
Abstract
The creation of high-fidelity, digital versions of human heads is an important stepping stone in the process of further integrating virtual components into our everyday lives. Constructing such avatars is a challenging research problem, due to a high demand for photo-realism and real-time rendering performance. In this work, we propose Neural Parametric Gaussian Avatars (NPGA), a data-driven approach to create high-fidelity, controllable avatars from multi-view video recordings. We build our method around 3D Gaussian splatting for its highly efficient rendering and to inherit the topological flexibility of point clouds. In contrast to previous work, we condition our avatars’ dynamics on the rich expression space of neural parametric head models (NPHM), instead of mesh-based 3DMMs. To this end, we distill the backward deformation field of our underlying NPHM into forward deformations which are compatible with rasterization-based rendering. All remaining fine-scale, expression-dependent details are learned from the multi-view videos. For increased representational capacity of our avatars, we propose per-Gaussian latent features that condition each primitives dynamic behavior. To regularize this increased dynamic expressivity, we propose Laplacian terms on the latent features and predicted dynamics. We evaluate our method on the public NeRSemble dataset, demonstrating that NPGA significantly outperforms the previous state-of-the-art avatars on the self-reenactment task by ≈ 2.6 PSNR. Furthermore, we demonstrate accurate animation capabilities from real-world monocular videos.
| Type: | Proceedings paper |
|---|---|
| Title: | NPGA: Neural Parametric Gaussian Avatars |
| Event: | SA '24: SIGGRAPH Asia 2024 Conference Papers |
| ISBN-13: | 9798400711312 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1145/3680528.368768 |
| Publisher version: | https://doi.org/10.1145/3680528.368768 |
| Language: | English |
| Additional information: | Copyright © 2024 Copyright held by the owner/author(s). This work is licensed under a Creative Commons Attribution International 4.0 License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. https://creativecommons.org/licenses/by/4.0/ |
| Keywords: | Virtual avatars, 3D Gaussian splatting, Data-driven animation, 3d morphable models |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10204042 |
Loading...
Loading...Loading...| 1. |  United Kingdom | 2 |
| 2. |  Norway | 1 |
Archive Staff Only
 |
View Item |
