Kokkinos, Filippos; Kokkinos, Iasonas; (2021) To The Point: Correspondence-driven monocular 3D category reconstruction. In: Ranzato, M and Beygelzimer, A and Dauphin, Y and Liang, PS and Wortman Vaughan, J, (eds.) Advances in Neural Information Processing Systems 34 (NeurIPS 2021). NeurIPS Proceedings: Virtual conference.

Preview

Text paper_ttp.pdf - Accepted Version Download (5MB) | Preview

Abstract

We present To The Point (TTP), a method for reconstructing 3D objects from a single image using 2D to 3D correspondences learned from weak supervision. We recover a 3D shape from a 2D image by first regressing the 2D positions corresponding to the 3D template vertices and then jointly estimating a rigid camera transform and non-rigid template deformation that optimally explain the 2D positions through the 3D shape projection. By relying on 3D-2D correspondences we use a simple per-sample optimization problem to replace CNN-based regression of camera pose and non-rigid deformation and thereby obtain substantially more accurate 3D reconstructions. We treat this optimization as a differentiable layer and train the whole system in an end-to-end manner. We report systematic quantitative improvements on multiple categories and provide qualitative results comprising diverse shape, pose and texture prediction examples. Project website: https://fkokkinos.github.io/to_the_point/

Download activity - last month

Export as JSONXMLCSV

Download activity - last 12 months

Export as JSONXMLCSV

Downloads by country - last 12 months

Export as XMLCSVJSON

Archive Staff Only

View Item