UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Inverse Graphics via Simulation-driven Learning

Innamorati, Carlo; (2021) Inverse Graphics via Simulation-driven Learning. Doctoral thesis (Ph.D), UCL (University College London). Green open access

[thumbnail of Inverse Graphics via Simulation-driven Learning.pdf]
Inverse Graphics via Simulation-driven Learning.pdf - Accepted Version

Download (310MB) | Preview


Bridging the gap between simulations and reality has always been an incredibly challenging task throughout a plethora of different fields. The rekindled advent of Neural Networks provided us with invaluable instruments for this purpose; at the same time, it brought along countless novel challenges. In this thesis, we pursue the optimization of these instruments for Computer Graphic tasks and, specifically, towards the generalization of synthetic solutions to reality. We tackle this goal through the innovation of state of the art Neural Network architectures and the design of learning-based systems, specifically built for the purpose of crossing the barrier between synthetic and real data. We explore two such solutions and lastly, tackle a related problem identified through the work pursued in the design of our workflows. Firstly, we work on a problem that is routinely faced by artists in product photography, where compromises are made to achieve higher artistic freedom: either composing multiple photos of the same scene, taken while varying the physical illumination, or to use synthetic (rendered) scenes. The project introduces a novel, carefully hand-crafted intrinsic decomposition, specifically enriched for the image manipulation workflow. It is then shown that the inverse of such a decomposition can be learned on synthetic data and be successfully generalized to real world images while retaining enough quality to perform image manipulation tasks that would otherwise require substantially more effort. Second, we introduce a method to generate videos of dynamic virtual objects plausibly interacting via collisions with a still image's environment. Given a starting trajectory, physically simulated with the estimated geometry of a single, static input image, we learn to 'correct' this trajectory to a visually plausible one via a neural network. Lastly, we provide insight into an inherent limitation of currently adopted padding solutions for Convolutional Neural Networks (CNNs). Specifically, CNNs handle the case where filters extend beyond the image boundary using several heuristics, such as zero, repeat or mean padding. These schemes, being weakly related to the image content and oblivious of the target task, result in low output quality at the boundary. We propose a simple and effective improvement, that learns the boundary handling itself. This is achieved by providing the network with a separate set of explicit boundary filters that can be seamlessly integrated.

Type: Thesis (Doctoral)
Qualification: Ph.D
Title: Inverse Graphics via Simulation-driven Learning
Event: UCL (University College London)
Open access status: An open access version is available from UCL Discovery
Language: English
Additional information: Copyright © The Author 2021. Original content in this thesis is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) Licence (https://creativecommons.org/licenses/by-nc/4.0/). Any third-party copyright material present remains the property of its respective owner(s) and is licensed under its existing terms. Access may initially be restricted at the author’s request.
UCL classification: UCL
UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10129626
Downloads since deposit
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item