UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Exploration, novelty, surprise, and free energy minimization.

Schwartenbeck, P; Fitzgerald, T; Dolan, RJ; Friston, K; (2013) Exploration, novelty, surprise, and free energy minimization. Front Psychol , 4 , Article 710. 10.3389/fpsyg.2013.00710. Green open access

[thumbnail of fpsyg-04-00710.pdf]
Preview
PDF
fpsyg-04-00710.pdf

Download (458kB)

Abstract

This paper reviews recent developments under the free energy principle that introduce a normative perspective on classical economic (utilitarian) decision-making based on (active) Bayesian inference. It has been suggested that the free energy principle precludes novelty and complexity, because it assumes that biological systems-like ourselves-try to minimize the long-term average of surprise to maintain their homeostasis. However, recent formulations show that minimizing surprise leads naturally to concepts such as exploration and novelty bonuses. In this approach, agents infer a policy that minimizes surprise by minimizing the difference (or relative entropy) between likely and desired outcomes, which involves both pursuing the goal-state that has the highest expected utility (often termed "exploitation") and visiting a number of different goal-states ("exploration"). Crucially, the opportunity to visit new states increases the value of the current state. Casting decision-making problems within a variational framework, therefore, predicts that our behavior is governed by both the entropy and expected utility of future states. This dissolves any dialectic between minimizing surprise and exploration or novelty seeking.

Type: Article
Title: Exploration, novelty, surprise, and free energy minimization.
Location: Switzerland
Open access status: An open access version is available from UCL Discovery
DOI: 10.3389/fpsyg.2013.00710
Publisher version: http://dx.doi.org/10.3389/fpsyg.2013.00710
Language: English
Additional information: © 2013 Schwartenbeck, FitzGerald, Dolan and Friston. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. PMCID: PMC3791848
Keywords: active inference, exploitation, exploration, free energy, novelty, reinforcement learning
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology > Imaging Neuroscience
URI: https://discovery.ucl.ac.uk/id/eprint/1423327
Downloads since deposit
228Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item