UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Active inference and learning

Friston, K; FitzGerald, T; Rigoli, F; Schwartenbeck, P; O'Doherty, J; Pezzulo, G; (2016) Active inference and learning. Neuroscience and Biobehavioral Reviews , 68 pp. 862-879. 10.1016/j.neubiorev.2016.06.022. Green open access

[thumbnail of 1-s2.0-S0149763416301336-main.pdf]
Preview
Text
1-s2.0-S0149763416301336-main.pdf

Download (5MB) | Preview

Abstract

This paper offers an active inference account of choice behaviour and learning. It focuses on the distinction between goal-directed and habitual behaviour and how they contextualise each other. We show that habits emerge naturally (and autodidactically) from sequential policy optimisation when agents are equipped with state-action policies. In active inference, behaviour has explorative (epistemic) and exploitative (pragmatic) aspects that are sensitive to ambiguity and risk respectively, where epistemic (ambiguity-resolving) behaviour enables pragmatic (reward-seeking) behaviour and the subsequent emergence of habits. Although goal-directed and habitual policies are usually associated with model-based and model-free schemes, we find the more important distinction is between belief-free and belief-based schemes. The underlying (variational) belief updating provides a comprehensive (if metaphorical) process theory for several phenomena, including the transfer of dopamine responses, reversal learning, habit formation and devaluation. Finally, we show that active inference reduces to a classical (Bellman) scheme, in the absence of ambiguity.

Type: Article
Title: Active inference and learning
Open access status: An open access version is available from UCL Discovery
DOI: 10.1016/j.neubiorev.2016.06.022
Publisher version: http://dx.doi.org/10.1016/j.neubiorev.2016.06.022
Language: English
Additional information: © 2016 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
Keywords: Active inference; Habit learning; Bayesian inference; Goal-directed; Free energy; Information gain; Bayesian surprise; Epistemic value; Exploration; Exploitation
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology > Imaging Neuroscience
URI: https://discovery.ucl.ac.uk/id/eprint/1506130
Downloads since deposit
109Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item