Innamorati, Carlo; (2021) Inverse Graphics via Simulation-driven Learning. Doctoral thesis (Ph.D), UCL (University College London).

Preview

Text Inverse Graphics via Simulation-driven Learning.pdf - Accepted Version Download (310MB) | Preview

Abstract

Bridging the gap between simulations and reality has always been an incredibly challenging task throughout a plethora of different fields. The rekindled advent of Neural Networks provided us with invaluable instruments for this purpose; at the same time, it brought along countless novel challenges. In this thesis, we pursue the optimization of these instruments for Computer Graphic tasks and, specifically, towards the generalization of synthetic solutions to reality. We tackle this goal through the innovation of state of the art Neural Network architectures and the design of learning-based systems, specifically built for the purpose of crossing the barrier between synthetic and real data. We explore two such solutions and lastly, tackle a related problem identified through the work pursued in the design of our workflows. Firstly, we work on a problem that is routinely faced by artists in product photography, where compromises are made to achieve higher artistic freedom: either composing multiple photos of the same scene, taken while varying the physical illumination, or to use synthetic (rendered) scenes. The project introduces a novel, carefully hand-crafted intrinsic decomposition, specifically enriched for the image manipulation workflow. It is then shown that the inverse of such a decomposition can be learned on synthetic data and be successfully generalized to real world images while retaining enough quality to perform image manipulation tasks that would otherwise require substantially more effort. Second, we introduce a method to generate videos of dynamic virtual objects plausibly interacting via collisions with a still image's environment. Given a starting trajectory, physically simulated with the estimated geometry of a single, static input image, we learn to 'correct' this trajectory to a visually plausible one via a neural network. Lastly, we provide insight into an inherent limitation of currently adopted padding solutions for Convolutional Neural Networks (CNNs). Specifically, CNNs handle the case where filters extend beyond the image boundary using several heuristics, such as zero, repeat or mean padding. These schemes, being weakly related to the image content and oblivious of the target task, result in low output quality at the boundary. We propose a simple and effective improvement, that learns the boundary handling itself. This is achieved by providing the network with a separate set of explicit boundary filters that can be seamlessly integrated.

Download activity - last month

Export as JSONCSVXML

Download activity - last 12 months

Export as JSONCSVXML

Downloads by country - last 12 months

Export as JSONCSVXML

Archive Staff Only

View Item