FitzGerald, TH;
Dolan, RJ;
Friston, K;
(2015)
Dopamine, reward learning, and active inference.
Frontiers in Computational Neuroscience
, 9
, Article 136. 10.3389/fncom.2015.00136.
Preview |
Text
Dopamine, reward learning, and active inference.pdf Download (3MB) | Preview |
Abstract
Temporal difference learning models propose phasic dopamine signaling encodes reward prediction errors that drive learning. This is supported by studies where optogenetic stimulation of dopamine neurons can stand in lieu of actual reward. Nevertheless, a large body of data also shows that dopamine is not necessary for learning, and that dopamine depletion primarily affects task performance. We offer a resolution to this paradox based on an hypothesis that dopamine encodes the precision of beliefs about alternative actions, and thus controls the outcome-sensitivity of behavior. We extend an active inference scheme for solving Markov decision processes to include learning, and show that simulated dopamine dynamics strongly resemble those actually observed during instrumental conditioning. Furthermore, simulated dopamine depletion impairs performance but spares learning, while simulated excitation of dopamine neurons drives reward learning, through aberrant inference about outcome states. Our formal approach provides a novel and parsimonious reconciliation of apparently divergent experimental findings.
| Type: | Article |
|---|---|
| Title: | Dopamine, reward learning, and active inference |
| Location: | Switzerland |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.3389/fncom.2015.00136 |
| Publisher version: | http://dx.doi.org/10.3389/fncom.2015.00136 |
| Language: | English |
| Additional information: | © 2015 FitzGerald, Dolan and Friston. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
| Keywords: | active inference, dopamine, incentive salience, instrumental conditioning, learning, reward, reward learning, variational inference |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology > Imaging Neuroscience |
| URI: | https://discovery.ucl.ac.uk/id/eprint/1474226 |
Archive Staff Only
 |
View Item |
