Morreale, Luca; Aigerman, Noam; Kim, Vladimir G; Mitra, Niloy J; (2024) Neural Semantic Surface Maps. Computer Graphics Forum , 43 (2) , Article e15005. 10.1111/cgf.15005.

Text 2309.04836v3.pdf - Accepted Version Access restricted to UCL open access staff until 18 April 2025. Download (28MB)

Abstract

We present an automated technique for computing a map between two genus‐zero shapes, which matches semantically corresponding regions to one another. Lack of annotated data prohibits direct inference of 3D semantic priors; instead, current state‐of‐the‐art methods predominantly optimize geometric properties or require varying amounts of manual annotation. To overcome the lack of annotated training data, we distill semantic matches from pre‐trained vision models: our method renders the pair of untextured 3D shapes from multiple viewpoints; the resulting renders are then fed into an off‐the‐shelf image‐matching strategy that leverages a pre‐trained visual model to produce feature points. This yields semantic correspondences, which are projected back to the 3D shapes, producing a raw matching that is inaccurate and inconsistent across different viewpoints. These correspondences are refined and distilled into an inter‐surface map by a dedicated optimization scheme, which promotes bijectivity and continuity of the output map. We illustrate that our approach can generate semantic surface‐to‐surface maps, eliminating manual annotations or any 3D training data requirement. Furthermore, it proves effective in scenarios with high semantic complexity, where objects are non‐isometrically related, as well as in situations where they are nearly isometric.

Download activity - last month

Export as XMLJSONCSV

Download activity - last 12 months

Export as XMLJSONCSV

Downloads by country - last 12 months

Export as CSVXMLJSON

Archive Staff Only

View Item