Li, Minne; (2022) Causal World Models. Doctoral thesis (Ph.D), UCL (University College London).

Preview

Text Minne_Li_Causal_World_Models_Final.pdf - Other Download (8MB) | Preview

Abstract

The capability of imagining internally with a mental model of the world is vitally important for human cognition. This mechanism has served as a core motivation behind world models (WMs), a family of sequential decision-making algorithms that predict the future state given an agent's historical experience and make optimal decisions through latent imagination accordingly. Conventional WMs build perception models based on partially observable Markov decision processes (POMDPs), which are often insufficient to handle information from varying state descriptions. Meanwhile, building WMs upon POMDPs usually separate perception modeling and decision-making, leaving the potential connection between the objectives of these processes unexplored. In addition, causal knowledge, which is critical for humans to simulate the alternative future that hasn't been experienced in the past, is missing from the current advancement of WMs. In this thesis, we will be considering an approach to address these issues by incorporating WMs with causal mechanisms. We firstly propose Causal World Models (CWMs), a general framework that learns behaviors purely from latent imagination of the future that would have happened. We then propose Multi-view World Models (MVWMs), which explicitly train for the ability to learn new concepts and shared dynamics of the environment from different state descriptions. Thirdly, we introduce a unification of Perception and Control as Inference (PCI) that facilitates a deep understanding of the mechanisms between perception and decision-making. We show that these novel WMs can be formulated together from a causality perspective. As a by-product of CWMs, we also show that modeling with causal structures can help tackle some of the key challenges in multi-agent systems and demonstrate the benefit through a real-world application. Finally, we conclude by discussing open questions and future directions in incorporating causality into reinforcement learning, aiming to identify the key shortcomings and limiting assumptions of our existing approaches. This thesis focuses on integrating causality into solving decision-making in partially observable environments, such as physical systems, humanoid robot control, and real-time ride-sharing order dispatching.

Download activity - last month

Export as XMLCSVJSON

Download activity - last 12 months

Export as XMLCSVJSON

Downloads by country - last 12 months

Export as JSONXMLCSV

Archive Staff Only

View Item