UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Policy adjustment in a dynamic economic game.

Li, J; McClure, SM; King-Casas, B; Montague, PR; (2006) Policy adjustment in a dynamic economic game. PLoS One , 1 (1) , Article e103. 10.1371/journal.pone.0000103. Green open access

[thumbnail of 1378380.pdf]

Download (459kB)


Making sequential decisions to harvest rewards is a notoriously difficult problem. One difficulty is that the real world is not stationary and the reward expected from a contemplated action may depend in complex ways on the history of an animal's choices. Previous functional neuroimaging work combined with principled models has detected brain responses that correlate with computations thought to guide simple learning and action choice. Those works generally employed instrumental conditioning tasks with fixed action-reward contingencies. For real-world learning problems, the history of reward-harvesting choices can change the likelihood of rewards collected by the same choices in the near-term future. We used functional MRI to probe brain and behavioral responses in a continuous decision-making task where reward contingency is a function of both a subject's immediate choice and his choice history. In these more complex tasks, we demonstrated that a simple actor-critic model can account for both the subjects' behavioral and brain responses, and identified a reward prediction error signal in ventral striatal structures active during these non-stationary decision tasks. However, a sudden introduction of new reward structures engages more complex control circuitry in the prefrontal cortex (inferior frontal gyrus and anterior insula) and is not captured by a simple actor-critic model. Taken together, these results extend our knowledge of reward-learning signals into more complex, history-dependent choice tasks. They also highlight the important interplay between striatum and prefrontal cortex as decision-makers respond to the strategic demands imposed by non-stationary reward environments more reminiscent of real-world tasks.

Type: Article
Title: Policy adjustment in a dynamic economic game.
Location: United States
Open access status: An open access version is available from UCL Discovery
DOI: 10.1371/journal.pone.0000103
Publisher version: http://dx.doi.org/10.1371/journal.pone.0000103
Language: English
Additional information: © 2006 Li et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. This work was funded by the Kane Family Foundation (P.R.M.), NINDS grant NS-045790 (P.R.M.) and NIDA grant DA-11723 (P.R.M.).
Keywords: Adult, Brain, Corpus Striatum, Decision Making, Female, Humans, Learning, Magnetic Resonance Imaging, Male, Models, Economic, Models, Neurological, Models, Psychological, Prefrontal Cortex, Reinforcement (Psychology), Reward, Young Adult
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology
URI: https://discovery.ucl.ac.uk/id/eprint/1378380
Downloads since deposit
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item