UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Efficiency and prioritization of inference-based credit assignment

Moran, R; Dayan, P; Dolan, RJ; (2021) Efficiency and prioritization of inference-based credit assignment. Current Biology 10.1016/j.cub.2021.03.091. (In press). Green open access

[thumbnail of Dolan_Efficiency and prioritization of inference-based credit assignment_AOP.pdf]
Preview
Text
Dolan_Efficiency and prioritization of inference-based credit assignment_AOP.pdf - Published Version

Download (1MB) | Preview

Abstract

Organisms adapt to their environments by learning to approach states that predict rewards and avoid states associated with punishments. Knowledge about the affective value of states often relies on credit assignment (CA), whereby state values are updated on the basis of reward feedback. Remarkably, humans assign credit to states that are not observed but are instead inferred based on a cognitive map that represents structural knowledge of an environment. A pertinent example is authors attempting to infer the identity of anonymous reviewers to assign them credit or blame and, on this basis, inform future referee recommendations. Although inference is cognitively costly, it is unknown how it influences CA or how it is apportioned between hidden and observable states (for example, both anonymous and revealed reviewers). We addressed these questions in a task that provided choices between lotteries where each led to a unique pair of occasionally rewarding outcome states. On some trials, both states were observable (rendering inference nugatory), whereas on others, the identity of one of the states was concealed. Importantly, by exploiting knowledge of choice-state associations, subjects could infer the identity of this hidden state. We show that having to perform inference reduces state-value updates. Strikingly, and in violation of normative theories, this reduction in CA was selective for the observed outcome alone. These findings have implications for the operation of putative cognitive maps.

Type: Article
Title: Efficiency and prioritization of inference-based credit assignment
Location: England
Open access status: An open access version is available from UCL Discovery
DOI: 10.1016/j.cub.2021.03.091
Publisher version: https://doi.org/10.1016/j.cub.2021.03.091
Language: English
Additional information: Copyright © 2021 The Author(s). Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
Keywords: cognitive maps, control, credit-assignment, decision making, inference, intrinsic value of information, model-based, model-free, reinforcement learning, reward
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology > Imaging Neuroscience
URI: https://discovery.ucl.ac.uk/id/eprint/10127294
Downloads since deposit
37Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item