Dutt, Niladri Shekhar;
Muralikrishnan, Sanjeev;
Mitra, Niloy J;
(2024)
Diffusion 3D Features (Diff3F) Decorating Untextured Shapes with Distilled Semantic Features.
In:
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024.
(pp. pp. 4494-4504).
Institute of Electrical and Electronics Engineers (IEEE)
Preview |
Text
2311.17024v2.pdf - Accepted Version Download (8MB) | Preview |
Abstract
We present Diff3F as a simple, robust, and class-agnostic feature descriptor that can be computed for untextured input shapes (meshes or point clouds). Our method distills diffusion features from image foundational models onto input shapes. Specifically, we use the input shapes to produce depth and normal maps as guidance for conditional image synthesis. In the process, we produce (diffusion) features in 2D that we subsequently lift and aggregate on the original surface. Our key observation is that even if the conditional image generations obtained from multi-view rendering of the input shapes are inconsistent, the associated image features are robust and, hence, can be directly aggregated across views. This produces semantic features on the input shapes, without requiring additional data or training. We perform extensive experiments on multiple benchmarks (SHREC'19, SHREC'20, FAUST, and TOSCA) and demonstrate that our features, being semantic instead of geometric, produce reliable correspondence across both isometric and non-isometrically related shape families. Code is available at https://github.com/niladridutt/Diffusion-3D-Features.
| Type: | Proceedings paper |
|---|---|
| Title: | Diffusion 3D Features (Diff3F) Decorating Untextured Shapes with Distilled Semantic Features |
| Event: | IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) |
| Location: | Seattle, WA, USA |
| Dates: | 16th-22nd June 2024 |
| ISBN-13: | 979-8-3503-5301-3 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/CVPR52733.2024.00430 |
| Publisher version: | https://doi.org/10.1109/cvpr52733.2024.00430 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher's terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10203446 |
Loading...
Loading...Loading...Loading...Archive Staff Only
 |
View Item |
