UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Model based planners reflect on their model-free propensities

Moran, R; Keramati, M; Dolan, RJ; (2021) Model based planners reflect on their model-free propensities. PLOS Computational Biology , 17 (1) , Article e1008552. 10.1371/journal.pcbi.1008552. Green open access

[thumbnail of Dolan_journal.pcbi.1008552.pdf]
Preview
Text
Dolan_journal.pcbi.1008552.pdf - Published Version

Download (1MB) | Preview

Abstract

Dual-reinforcement learning theory proposes behaviour is under the tutelage of a retrospective, value-caching, model-free (MF) system and a prospective-planning, model-based (MB), system. This architecture raises a question as to the degree to which, when devising a plan, a MB controller takes account of influences from its MF counterpart. We present evidence that such a sophisticated self-reflective MB planner incorporates an anticipation of the influences its own MF-proclivities exerts on the execution of its planned future actions. Using a novel bandit task, wherein subjects were periodically allowed to design their environment, we show that reward-assignments were constructed in a manner consistent with a MB system taking account of its MF propensities. Thus, in the task participants assigned higher rewards to bandits that were momentarily associated with stronger MF tendencies. Our findings have implications for a range of decision making domains that includes drug abuse, pre-commitment, and the tension between short and long-term decision horizons in economics.

Type: Article
Title: Model based planners reflect on their model-free propensities
Location: United States
Open access status: An open access version is available from UCL Discovery
DOI: 10.1371/journal.pcbi.1008552
Publisher version: https://doi.org/10.1371/journal.pcbi.1008552
Language: English
Additional information: © 2021 Moran et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology > Imaging Neuroscience
URI: https://discovery.ucl.ac.uk/id/eprint/10119020
Downloads since deposit
36Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item