Luo, Rui; (2024) Approximate Inference on Structured Distributions using Stochastic Dynamics. Doctoral thesis (Ph.D), UCL (University College London).

Preview

Text Luo_10194414_thesis.pdf Download (12MB) | Preview

Abstract

In this dissertation, we study approximate inference with an emphasis on practical methods for efficient inference in respect of complex distributions, typically those with highly non-convex geometry or multiscale temporal dependencies. In particular, we tackle two specific problems arising from the application of sophisticated deep neural architectures and structured data on very large scales, presenting constructive solutions incorporating stochastic dynamics as well as state-of-the-art techniques originated in physics and statistics. We resolve the first problem of effectively generating representative samples from a posterior distribution which potentially exhibits a composite surface composed of multiple, isolated modes, by presenting two advanced Markov chain Monte Carlo algorithms that leverage the Nosé-Hoover thermostat and techniques of continuous or parallel tempering for different use cases. We improve the Hamiltonian system of continuous tempering with the Nosé-Hoover thermostat, devising a continuouslytempered Nosé-Hoover dynamics allowing for fast traversing multimodal density landscapes and adaptively dissipating random noise within stochastic gradient. In scenarios of very large datasets, we propose alternatively a scalable replica-exchange method equipped with a tailored, noise-resistant protocol of exchange, simulating an ensemble of instances of the Nosé-Hoover dynamics in parallel and probabilistically swapping the states of instances to accelerate exploration of the state space with variable curvature. The second challengewe address is building and efficient learning of flexible statistical models representing the time evolution of latent stochastic dynamics underlying various types of stochastic volatility. We design a probabilistic graphical model as a general framework unifying a wide range of formulations of stochastic volatility, and develop a variational auto-encoding architecture that implements the graphical model with neural recurrent units, facilitating inference on the posterior distribution associated with the latent dynamics.

Download activity - last month

Export as CSVJSONXML

Download activity - last 12 months

Export as JSONXMLCSV

Downloads by country - last 12 months

Export as JSONXMLCSV

Archive Staff Only

View Item