UCL logo

UCL Discovery

UCL home » Library Services » Electronic resources » UCL Discovery

Effcient inference in Markov control problems

Furmston, T; Barber, D; (2011) Effcient inference in Markov control problems. Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence, UAI 2011 pp. 221-229.

Full text not available from this repository.


Markov control algorithms that perform smooth, non-greedy updates of the policy have been shown to be very general and versatile, with policy gradient and Expectation Maximisation algorithms being particularly popular. For these algorithms, marginal inference of the reward weighted trajectory distribution is required to perform policy updates. We discuss a new exact inference algorithm for these marginals in the finite horizon case that is more effcient than the standard approach based on classical forwardbackward recursions. We also provide a principled extension to infinite horizon Markov Decision Problems that explicitly accounts for an infinite horizon. This extension provides a novel algorithm for both policy gradients and Expectation Maximisation in infinite horizon problems.

Type: Article
Title: Effcient inference in Markov control problems
UCL classification: UCL > School of BEAMS
UCL > School of BEAMS > Faculty of Engineering Science
URI: http://discovery.ucl.ac.uk/id/eprint/1366371
Downloads since deposit
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item