eprintid: 10191492
rev_number: 10
eprint_status: archive
userid: 699
dir: disk0/10/19/14/92
datestamp: 2024-05-01 08:01:14
lastmod: 2024-10-08 13:21:33
status_changed: 2024-05-01 08:01:14
type: article
metadata_visibility: show
sword_depositor: 699
creators_name: Morreale, Luca
creators_name: Aigerman, Noam
creators_name: Kim, Vladimir G
creators_name: Mitra, Niloy J
title: Neural Semantic Surface Maps
ispublished: pub
divisions: UCL
divisions: B04
divisions: C05
divisions: F48
keywords: CCS Concepts, Computing methodologies, Shape analysis, Mesh geometry models, Feature selection
note: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
abstract: We present an automated technique for computing a map between two genus‐zero shapes, which matches semantically corresponding regions to one another. Lack of annotated data prohibits direct inference of 3D semantic priors; instead, current state‐of‐the‐art methods predominantly optimize geometric properties or require varying amounts of manual annotation. To overcome the lack of annotated training data, we distill semantic matches from pre‐trained vision models: our method renders the pair of untextured 3D shapes from multiple viewpoints; the resulting renders are then fed into an off‐the‐shelf image‐matching strategy that leverages a pre‐trained visual model to produce feature points. This yields semantic correspondences, which are projected back to the 3D shapes, producing a raw matching that is inaccurate and inconsistent across different viewpoints. These correspondences are refined and distilled into an inter‐surface map by a dedicated optimization scheme, which promotes bijectivity and continuity of the output map. We illustrate that our approach can generate semantic surface‐to‐surface maps, eliminating manual annotations or any 3D training data requirement. Furthermore, it proves effective in scenarios with high semantic complexity, where objects are non‐isometrically related, as well as in situations where they are nearly isometric.
date: 2024-05
date_type: published
publisher: Wiley
official_url: http://dx.doi.org/10.1111/cgf.15005
full_text_type: other
language: eng
verified: verified_manual
elements_id: 2270732
doi: 10.1111/cgf.15005
lyricists_name: Mitra, Niloy
lyricists_id: NMITR19
actors_name: Flynn, Bernadette
actors_id: BFFLY94
actors_role: owner
full_text_status: restricted
publication: Computer Graphics Forum
volume: 43
number: 2
article_number: e15005
citation:        Morreale, Luca;    Aigerman, Noam;    Kim, Vladimir G;    Mitra, Niloy J;      (2024)    Neural Semantic Surface Maps.                   Computer Graphics Forum , 43  (2)    , Article e15005.  10.1111/cgf.15005 <https://doi.org/10.1111/cgf.15005>.      
 
document_url: https://discovery.ucl.ac.uk/id/eprint/10191492/1/2309.04836v3.pdf