Schmitt, S; Shawe-Taylor, J; van Hasselt, H; (2023) Exploration via Epistemic Value Estimation. In: Proceedings of the 37th AAAI Conference on Artificial Intelligence, AAAI 2023. (pp. pp. 9742-9751). The Association for the Advancement of Artificial Intelligence (AAAI)

Preview

Text Shawe-Taylo_Exploration via Epistemic Value Estimation_AAM.pdf Download (691kB) | Preview

Abstract

How to efficiently explore in reinforcement learning is an open problem. Many exploration algorithms employ the epistemic uncertainty of their own value predictions - for instance to compute an exploration bonus or upper confidence bound. Unfortunately the required uncertainty is difficult to estimate in general with function approximation. We propose epistemic value estimation (EVE): a recipe that is compatible with sequential decision making and with neural network function approximators. It equips agents with a tractable posterior over all their parameters from which epistemic value uncertainty can be computed efficiently. We use the recipe to derive an epistemic Q-Learning agent and observe competitive performance on a series of benchmarks. Experiments confirm that the EVE recipe facilitates efficient exploration in hard exploration tasks.

Download activity - last month

Export as CSVJSONXML

Download activity - last 12 months

Export as JSONCSVXML

Downloads by country - last 12 months

Export as XMLJSONCSV

Archive Staff Only

View Item