Walker, William; (2024) Probabilistic Unsupervised Learning using Recognition Parameterized Models. Doctoral thesis (Ph.D), UCL (University College London).

Preview

Text William_Walker_Thesis_Final.pdf - Accepted Version Download (8MB) | Preview

Abstract

Animals and some artificial agents must use sensory information about the environment to infer representations that allow for further action selection and cognition. This sensory information often is unlabelled and latent statistical structure must be found in noisy observations. The recognition-parametrized model (RPM) is a novel approach to such probabilistic unsupervised learning that incorporates a flexible recognition model, giving posteriors on latent variables, without the need for a separate generate model. As a normalized semi-parametric hypothesis class for joint distributions over observed and latent variables, the RPM parameterizes both the prior distribution on the latents and their conditional distributions given observations under the key assumption that observations are conditionally independent given the latents. The recognition model is paired with non-parametric descriptions of the marginal distribution of each observed variable. Thus, the focus is on learning a good latent representation that captures dependence between the measurements. The RPM permits exact maximum-likelihood learning in settings with discrete latents and a tractable prior, even when the mapping between continuous observations and the latents is expressed through a flexible model such as a neural network. We develop effective approximations for the case of continuous latent variables with tractable priors. Unlike the approximations necessary in dual-parametrized models such as Helmholtz machines and variational autoencoders, these RPM approximations introduce only minor bias, which may often vanish asymptotically. Furthermore, where the prior on latents is intractable the RPM may be combined effectively with standard probabilistic techniques such as variational Bayes. The RPM provides an effective way to discover, represent and reason probabilistically about the latent structure underlying observational data, functions which are critical to both animal and artificial intelligence.

Download activity - last month

Export as JSONCSVXML

Download activity - last 12 months

Export as JSONCSVXML

Downloads by country - last 12 months

Export as XMLCSVJSON

Archive Staff Only

View Item