UCL logo

UCL Discovery

UCL home » Library Services » Electronic resources » UCL Discovery

Variational methods for reinforcement learning

Furmston, T; Barber, D; (2010) Variational methods for reinforcement learning. Journal of Machine Learning Research , 9 pp. 241-248. Gold open access


We consider reinforcement learning as solving a Markov decision process with unknown transition distribution. Based on interaction with the environment, an estimate of the transition matrix is obtained from which the optimal decision policy is formed. The classical maximum likelihood point estimate of the transition model does not reflect the uncertainty in the estimate of the transition model and the resulting policies may consequently lack a sufficient degree of exploration. We consider a Bayesian alternative that maintains a distribution over the transition so that the resulting policy takes into account the limited experience of the environment. The resulting algorithm is formally intractable and we discuss two approximate solution methods, Variational Bayes and Expectation Propagation. Copyright 2010 by the authors.

Type: Article
Title: Variational methods for reinforcement learning
Open access status: An open access publication
UCL classification: UCL > School of BEAMS
UCL > School of BEAMS > Faculty of Engineering Science
URI: http://discovery.ucl.ac.uk/id/eprint/1366372
Downloads since deposit
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item