UCL logo

UCL Discovery

UCL home » Library Services » Electronic resources » UCL Discovery

Markov chain Monte Carlo for continuous-time discrete-state systems

Rao, VAP; (2012) Markov chain Monte Carlo for continuous-time discrete-state systems. Doctoral thesis, UCL (University College London). Green open access

PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader


A variety of phenomena are best described using dynamical models which operate on a discrete state space and in continuous time. Examples include Markov (and semi-Markov) jump processes, continuous-time Bayesian networks, renewal processes and other point processes. These continuous-time, discrete-state models are ideal building blocks for Bayesian models in fields such as systems biology, genetics, chemistry, computing networks, human-computer interactions etc. However, a challenge towards their more widespread use is the computational burden of posterior inference; this typically involves approximations like time discretization and can be computationally intensive. In this thesis, we describe a new class of Markov chain Monte Carlo methods that allow efficient computation while still being exact. The core idea is an auxiliary variable Gibbs sampler that alternately resamples a random discretization of time given the state-trajectory of the system, and then samples a new trajectory given this discretization. We introduce this idea by relating it to a classical idea called uniformization, and use it to develop algorithms that outperform the state-of-the-art for models based on the Markov jump process. We then extend the scope of these samplers to a wider class of models such as nonstationary renewal processes, and semi-Markov jump processes. By developing a more general framework beyond uniformization, we remedy various limitations of the original algorithms, allowing us to develop MCMC samplers for systems with infinite state spaces, unbounded rates, as well as systems indexed by more general continuous spaces than time.

Type:Thesis (Doctoral)
Title:Markov chain Monte Carlo for continuous-time discrete-state systems
Open access status:An open access version is available from UCL Discovery
Keywords:Markov chain Monte Carlo, continuous-time, Markov jump process, semi Markov process, uniformization, forward-backward algorithm
UCL classification:UCL > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neuroscience Unit

View download statistics for this item

Archive Staff Only: edit this record