UCL logo

UCL Discovery

UCL home » Library Services » Electronic resources » UCL Discovery

Variational methods for reinforcement learning

Furmston, T; Barber, D; (2010) Variational methods for reinforcement learning. Journal of Machine Learning Research , 9 241 - 248. Gold open access


We consider reinforcement learning as solving a Markov decision process with unknown transition distribution. Based on interaction with the environment, an estimate of the transition matrix is obtained from which the optimal decision policy is formed. The classical maximum likelihood point estimate of the transition model does not reflect the uncertainty in the estimate of the transition model and the resulting policies may consequently lack a sufficient degree of exploration. We consider a Bayesian alternative that maintains a distribution over the transition so that the resulting policy takes into account the limited experience of the environment. The resulting algorithm is formally intractable and we discuss two approximate solution methods, Variational Bayes and Expectation Propagation. Copyright 2010 by the authors.

Title:Variational methods for reinforcement learning
Open access status:An open access publication
Publisher version:http://www.jmlr.org/
UCL classification:UCL > School of BEAMS > Faculty of Engineering Science > Computer Science

Archive Staff Only: edit this record