Xu, Yang;
Yan, Yan;
Xue, Jing-Hao;
Hua, Yang;
Wang, Hanzi;
(2024)
Unpaired Caricature-Visual Face Recognition via Feature Decomposition-Restoration-Decomposition.
IEEE Transactions on Circuits and Systems for Video Technology
10.1109/tcsvt.2024.3361799.
(In press).
Preview |
Text
Xue_Unpaired Caricature-Visual Face Recognition via Feature Decomposition-Restoration-Decomposition_AAM2.pdf Download (1MB) | Preview |
Abstract
Existing caricature-visual face recognition methods train the models based on caricature-visual image pairs from the same identities. Unfortunately, in many real-world applications, facial caricatures and visual facial images are usually unpaired in the training set due to the difficulty of collecting facial caricatures drawn by artists. In this paper, we study caricature-visual face recognition under the practical setting that only unpaired facial caricature and visual facial images are available as training samples, and define this setting as unpaired caricature-visual face recognition. To this end, we develop a novel feature decomposition-restoration-decomposition method (FDRD), which mainly consists of a backbone network, an identity-oriented feature decomposition module, and a modality-oriented feature restoration module, to extract modality-irrelevant identity features. To effectively train FDRD in the case of limited facial caricature training samples, we develop a two-stage learning framework. In the first stage, we perform single-modality restoration, enabling the model to have the basic ability of feature decomposition and restoration for each modality. In the second stage, we perform cross-modality recognition by exchanging new modality features between the two modalities, facilitating the model to focus on the decoupling of identity features and modality features. Experimental results demonstrate that our method performs favorably against several state-of-the-art face recognition methods and cross-modality methods. Our code is available at https://github.com/Capricorn-Karma/FDRD.
| Type: | Article |
|---|---|
| Title: | Unpaired Caricature-Visual Face Recognition via Feature Decomposition-Restoration-Decomposition |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/tcsvt.2024.3361799 |
| Publisher version: | http://dx.doi.org/10.1109/tcsvt.2024.3361799 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher's terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences > Dept of Statistical Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10187013 |
Archive Staff Only
 |
View Item |
