Henzler, Philipp; Reizenstein, Jeremy; Labatut, Patrick; Shapovalov, Roman; Ritschel, Tobias; Vedaldi, Andrea; Novotny, David; (2021) Unsupervised Learning of 3D Object Categories from Videos in the Wild. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). (pp. pp. 4698-4707). IEEE: Nashville, TN, USA.

Preview

PDF Henzler_Unsupervised_Learning_of_3D_Object_Categories_From_Videos_in_the_CVPR_2021_paper.pdf - Accepted Version Download (2MB) | Preview

Abstract

Our goal is to learn a deep network that, given a small number of images of an object of a given category, reconstructs it in 3D. While several recent works have obtained analogous results using synthetic data or assuming the avail-ability of 2D primitives such as keypoints, we are interested in working with challenging real data and with no manual annotations. We thus focus on learning a model from multiple views of a large collection of object instances. We contribute with a new large dataset of object centric videos suitable for training and benchmarking this class of models. We show that existing techniques leveraging meshes, voxels, or implicit surfaces, which work well for reconstructing isolated objects, fail on this challenging data. Finally, we propose a new neural network design, called warp-conditioned ray embedding (WCR), which significantly improves reconstruction while obtaining a detailed implicit representation of the object surface and texture, also compensating for the noise in the initial SfM reconstruction that bootstrapped the learning process. Our evaluation demonstrates performance improvements over several deep monocular reconstruction baselines on existing benchmarks and on our novel dataset. For additional material please visit: https://henzler.github.io/publication/unsupervised_videos/.

Download activity - last month

Export as JSONCSVXML

Download activity - last 12 months

Export as XMLCSVJSON

Downloads by country - last 12 months

Export as JSONCSVXML

Archive Staff Only

View Item