Kokkinos, F; Kokkinos, I; (2021) Learning monocular 3D reconstruction of articulated categories from motion. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). (pp. pp. 1737-1746). IEEE: Nashville, TN, USA.

Preview

PDF paper.pdf - Accepted Version Download (9MB) | Preview

Abstract

Monocular 3D reconstruction of articulated object categories is challenging due to the lack of training data and the inherent ill-posedness of the problem. In this work we use video self-supervision, forcing the consistency of consecutive 3D reconstructions by a motion-based cycle loss. This largely improves both optimization-based and learning-based 3D mesh reconstruction. We further introduce an interpretable model of 3D template deformations that controls a 3D surface through the displacement of a small number of local, learnable handles. We formulate this operation as a structured layer relying on mesh-laplacian regularization and show that it can be trained in an end-to-end manner. We finally introduce a per-sample numerical optimisation approach that jointly optimises over mesh displacements and cameras within a video, boosting accuracy both for training and also as test time post-processing. While relying exclusively on a small set of videos collected per category for supervision, we obtain state-of-the-art reconstructions with diverse shapes, viewpoints and textures for multiple articulated object categories. Supplementary materials, code, and videos are provided on the project page: https://fkokkinos.github.io/video_3d_reconstruction/.

Downloads since deposit

Loading...

49Downloads

Download activity - last month

Export as JSONXMLCSV

Loading...

Download activity - last 12 months

Export as JSONCSVXML

Loading...

Downloads by country - last 12 months

Export as CSVXMLJSON

Loading...

Archive Staff Only

View Item