Lu, Jiawei;
Shao, Tianjia;
Wang, He;
Yang, Yong-Liang;
Yang, Yin;
Zhou, Kun;
(2024)
Relightable Detailed Human Reconstruction from Sparse Flashlight Images.
IEEE Transactions on Visualization and Computer Graphics
pp. 1-13.
10.1109/tvcg.2024.3450591.
(In press).
Preview |
Text
Wang_Relightable Detailed Human Reconstruction from Sparse Flashlight Images_AAM.pdf Download (22MB) | Preview |
Abstract
We present a lightweight system for reconstructing human geometry and appearance from sparse flashlight images. Our system produces detailed geometry including garment wrinkles and surface reflectance, which are exportable for direct rendering and relighting in traditional graphics pipelines. By capturing multi-view flashlight images using a consumer camera equipped with an co-located LED (e.g., a cell phone), we obtain view-specific shading cues that aid in the determination of surface orientation and help disambiguate between shading and material. To enable the reconstruction of geometry and appearance from sparse-view flashlight images, we integrate a pre-trained model into a differentiable physics-based rendering framework. As the learned image features from synthetic data cannot accurately reflect the shading features on real images, which is crucial for the high-quality reconstruction of geometry details and appearance, we propose to jointly optimize the image feature extractor with two MLPs for SDF and BRDF prediction using the differentiable physics-based rendering. Compared with existing methods for relightable human reconstruction, our system is able to produce high-fidelity 3D human models with more accurate geometry and appearance under the same condition. Our code and data are available at http://github.com/Jarvisss/Relightable_human_recon .
| Type: | Article |
|---|---|
| Title: | Relightable Detailed Human Reconstruction from Sparse Flashlight Images |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/tvcg.2024.3450591 |
| Publisher version: | http://dx.doi.org/10.1109/tvcg.2024.3450591 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher's terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10196958 |
Loading...
Loading...Loading...Loading...Archive Staff Only
 |
View Item |
