eprintid: 10191492 rev_number: 10 eprint_status: archive userid: 699 dir: disk0/10/19/14/92 datestamp: 2024-05-01 08:01:14 lastmod: 2024-10-08 13:21:33 status_changed: 2024-05-01 08:01:14 type: article metadata_visibility: show sword_depositor: 699 creators_name: Morreale, Luca creators_name: Aigerman, Noam creators_name: Kim, Vladimir G creators_name: Mitra, Niloy J title: Neural Semantic Surface Maps ispublished: pub divisions: UCL divisions: B04 divisions: C05 divisions: F48 keywords: CCS Concepts, Computing methodologies, Shape analysis, Mesh geometry models, Feature selection note: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. abstract: We present an automated technique for computing a map between two genus‐zero shapes, which matches semantically corresponding regions to one another. Lack of annotated data prohibits direct inference of 3D semantic priors; instead, current state‐of‐the‐art methods predominantly optimize geometric properties or require varying amounts of manual annotation. To overcome the lack of annotated training data, we distill semantic matches from pre‐trained vision models: our method renders the pair of untextured 3D shapes from multiple viewpoints; the resulting renders are then fed into an off‐the‐shelf image‐matching strategy that leverages a pre‐trained visual model to produce feature points. This yields semantic correspondences, which are projected back to the 3D shapes, producing a raw matching that is inaccurate and inconsistent across different viewpoints. These correspondences are refined and distilled into an inter‐surface map by a dedicated optimization scheme, which promotes bijectivity and continuity of the output map. We illustrate that our approach can generate semantic surface‐to‐surface maps, eliminating manual annotations or any 3D training data requirement. Furthermore, it proves effective in scenarios with high semantic complexity, where objects are non‐isometrically related, as well as in situations where they are nearly isometric. date: 2024-05 date_type: published publisher: Wiley official_url: http://dx.doi.org/10.1111/cgf.15005 full_text_type: other language: eng verified: verified_manual elements_id: 2270732 doi: 10.1111/cgf.15005 lyricists_name: Mitra, Niloy lyricists_id: NMITR19 actors_name: Flynn, Bernadette actors_id: BFFLY94 actors_role: owner full_text_status: restricted publication: Computer Graphics Forum volume: 43 number: 2 article_number: e15005 citation: Morreale, Luca; Aigerman, Noam; Kim, Vladimir G; Mitra, Niloy J; (2024) Neural Semantic Surface Maps. Computer Graphics Forum , 43 (2) , Article e15005. 10.1111/cgf.15005 <https://doi.org/10.1111/cgf.15005>. document_url: https://discovery.ucl.ac.uk/id/eprint/10191492/1/2309.04836v3.pdf