Karnewar, A;
Shapovalov, R;
Monnier, T;
Vedaldi, A;
Mitra, NJ;
Novotny, D;
(2025)
GOEmbed: Gradient Origin Embeddings for Representation Agnostic 3D Feature Learning.
In: Leonardis, A and Ricci, E and Roth, S and Russakovsky, O and Sattler, T and Varol, G, (eds.)
Computer Vision – ECCV 2024.
(pp. pp. 454-472).
Springer: Cham, Switzerland.
![[thumbnail of karnewar2023goembed.pdf]](https://discovery.ucl.ac.uk/style/images/fileicons/text.png) |
Text
karnewar2023goembed.pdf - Published Version Access restricted to UCL open access staff until 9 November 2025. Download (34MB) |
Abstract
Encoding information from 2D views of an object into a 3D representation is crucial for generalized 3D feature extraction. Such features can then enable 3D reconstruction, 3D generation, and other applications. We propose GOEmbed (Gradient Origin Embeddings) that encodes input 2D images into any 3D representation, without requiring a pre-trained image feature extractor; unlike typical prior approaches in which input images are either encoded using 2D features extracted from large pre-trained models, or customized features are designed to handle different 3D representations; or worse, encoders may not yet be available for specialized 3D neural representations such as MLPs and hash-grids. We extensively evaluate our proposed GOEmbed under different experimental settings on the OmniObject3D benchmark. First, we evaluate how well the mechanism compares against prior encoding mechanisms on multiple 3D representations using an illustrative experiment called Plenoptic-Encoding. Second, the efficacy of the GOEmbed mechanism is further demonstrated by achieving a new SOTA FID of 22.12 on the OmniObject3D generation task using a combination of GOEmbed and DFM (Diffusion with Forward Models), which we call GOEmbedFusion. Finally, we evaluate how the GOEmbed mechanism bolsters sparse-view 3D reconstruction pipelines.
| Type: | Proceedings paper |
|---|---|
| Title: | GOEmbed: Gradient Origin Embeddings for Representation Agnostic 3D Feature Learning |
| Event: | Computer Vision – ECCV 2024 (ECCV 2024) |
| Location: | ITALY |
| Dates: | 29 Sep 2024 - 4 Oct 2024 |
| ISBN-13: | 978-3-031-73222-5 |
| DOI: | 10.1007/978-3-031-73223-2_25 |
| Publisher version: | https://doi.org/10.1007/978-3-031-73223-2_25 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | Science & Technology, Technology, Computer Science, Artificial Intelligence, Computer Science, Interdisciplinary Applications, Computer Science, Theory & Methods, Computer Science |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10204220 |
Loading...
Loading...Loading...Loading...Archive Staff Only
 |
View Item |
