UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Active Inference, epistemic value, and vicarious trial and error

Pezzulo, G; Cartoni, E; Rigoli, F; Pio-Lopez, L; Friston, K; (2016) Active Inference, epistemic value, and vicarious trial and error. LEARNING & MEMORY , 23 (7) pp. 322-338. 10.1101/lm.041780.116. Green open access

[img]
Preview
Text
Learn. Mem.-2016-Pezzulo-322-38.pdf - Published version

Download (2MB) | Preview

Abstract

Balancing habitual and deliberate forms of choice entails a comparison of their respective merits—the former being faster but inflexible, and the latter slower but more versatile. Here, we show that arbitration between these two forms of control can be derived from first principles within an Active Inference scheme. We illustrate our arguments with simulations that reproduce rodent spatial decisions in T-mazes. In this context, deliberation has been associated with vicarious trial and error (VTE) behavior (i.e., the fact that rodents sometimes stop at decision points as if deliberating between choice alternatives), whose neurophysiological correlates are “forward sweeps” of hippocampal place cells in the arms of the maze under consideration. Crucially, forward sweeps arise early in learning and disappear shortly after, marking a transition from deliberative to habitual choice. Our simulations show that this transition emerges as the optimal solution to the trade-off between policies that maximize reward or extrinsic value (habitual policies) and those that also consider the epistemic value of exploratory behavior (deliberative or epistemic policies)—the latter requiring VTE and the retrieval of episodic information via forward sweeps. We thus offer a novel perspective on the optimality principles that engender forward sweeps and VTE, and on their role on deliberate choice.

Type: Article
Title: Active Inference, epistemic value, and vicarious trial and error
Open access status: An open access version is available from UCL Discovery
DOI: 10.1101/lm.041780.116
Publisher version: http://dx.doi.org/10.1101/lm.041780.116
Language: English
Additional information: © 2016 Pezzulo et al.; Published by Cold Spring Harbor Laboratory Press
Keywords: Science & Technology, Social Sciences, Life Sciences & Biomedicine, Neurosciences, Psychology, Experimental, Neurosciences & Neurology, Psychology, HIPPOCAMPAL-THETA SEQUENCES, GOAL-DIRECTED BEHAVIOR, PLACE-CELL SEQUENCES, DECISION-MAKING, COGNITIVE CONTROL, DORSAL STRIATUM, BRAIN, INFORMATION, EXPLORATION, SYSTEMS
UCL classification: UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology > Department of Neuromuscular Diseases
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology > Imaging Neuroscience
URI: https://discovery.ucl.ac.uk/id/eprint/1502685
Downloads since deposit
101Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item