UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Credit assignment to state-independent task representations and its relationship with model-based decision making

Shahar, N; Moran, R; Hauser, TU; Kievit, RA; McNamee, D; Moutoussis, M; NSPN Consortium, .; (2019) Credit assignment to state-independent task representations and its relationship with model-based decision making. Proceedings of the National Academy of Sciences 10.1073/pnas.1821647116. (In press). Green open access

[img]
Preview
Text
Dolan_Credit assignment to state-independent task representations and its relationship with model-based decision making_VoR.pdf - Published version

Download (778kB) | Preview

Abstract

Model-free learning enables an agent to make better decisions based on prior experience while representing only minimal knowledge about an environment’s structure. It is generally assumed that model-free state representations are based on outcome-relevant features of the environment. Here, we challenge this assumption by providing evidence that a putative model-free system assigns credit to task representations that are irrelevant to an outcome. We examined data from 769 individuals performing a well-described 2-step reward decision task where stimulus identity but not spatial-motor aspects of the task predicted reward. We show that participants assigned value to spatial-motor representations despite it being outcome irrelevant. Strikingly, spatial-motor value associations affected behavior across all outcome-relevant features and stages of the task, consistent with credit assignment to low-level state-independent task representations. Individual difference analyses suggested that the impact of spatial-motor value formation was attenuated for individuals who showed greater deployment of goal-directed (model-based) strategies. Our findings highlight a need for a reconsideration of how model-free representations are formed and regulated according to the structure of the environment.

Type: Article
Title: Credit assignment to state-independent task representations and its relationship with model-based decision making
Location: United States
Open access status: An open access version is available from UCL Discovery
DOI: 10.1073/pnas.1821647116
Publisher version: https://doi.org/10.1073/pnas.1821647116
Language: English
Additional information: © 2019 the Author(s). Published by PNAS. This open access article is distributed under Creative Commons Attribution License 4.0 (http://creativecommons.org/licenses/by/4.0/).
Keywords: decision making, motor learning, reinforcement learning
UCL classification: UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology > Imaging Neuroscience
URI: https://discovery.ucl.ac.uk/id/eprint/10078943
Downloads since deposit
18Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item