Karnewar, A;
Mitra, NJ;
Vedaldi, A;
Novotny, D;
(2024)
HoloFusion: Towards Photo-realistic 3D Generative Modeling.
In:
Proceedings of the IEEE International Conference on Computer Vision.
(pp. pp. 22919-22928).
Institute of Electrical and Electronics Engineers (IEEE)
Preview |
Text
2308.14244.pdf - Accepted Version Download (31MB) | Preview |
Abstract
Diffusion-based image generators can now produce high-quality and diverse samples, but their success has yet to fully translate to 3D generation: existing diffusion methods can either generate low-resolution but 3D consistent outputs, or detailed 2D views of 3D objects but with potential structural defects and lacking view consistency or realism. We present HoloFusion, a method that combines the best of these approaches to produce high-fidelity, plausible, and diverse 3D samples while learning from a collection of multi-view 2D images only. The method first generates coarse 3D samples using a variant of the recently proposed HoloDiffusion generator. Then, it independently renders and upsamples a large number of views of the coarse 3D model, super-resolves them to add detail, and distills those into a single, high-fidelity implicit 3D representation, which also ensures view-consistency of the final renders. The super-resolution network is trained as an integral part of HoloFusion, end-to-end, and the final distillation uses a new sampling scheme to capture the space of super-resolved signals. We compare our method against existing baselines, including DreamFusion, Get3D, EG3D, and HoloDiffusion, and achieve, to the best of our knowledge, the most realistic results on the challenging CO3Dv2 dataset.
| Type: | Proceedings paper |
|---|---|
| Title: | HoloFusion: Towards Photo-realistic 3D Generative Modeling |
| Event: | 2023 IEEE/CVF International Conference on Computer Vision (ICCV) |
| Location: | Paris, France |
| Dates: | 1st-6th October 2023 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/ICCV51070.2023.02100 |
| Publisher version: | http://dx.doi.org/10.1109/iccv51070.2023.02100 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher's terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10189973 |
Loading...
Loading...Loading...Loading...Archive Staff Only
 |
View Item |
