UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Adaptive integration of habits into depth-limited planning defines a habitual-goal-directed spectrum

Keramati, M; Smittenaar, P; Dolan, RJ; Dayan, P; (2016) Adaptive integration of habits into depth-limited planning defines a habitual-goal-directed spectrum. Proceedings of The National Academy of Sciences of The United States of America , 113 (45) pp. 12868-12873. 10.1073/pnas.1609094113. Green open access

[img]
Preview
Text
Keramati_Keramati et al 2016.pdf - Accepted version

Download (9MB) | Preview

Abstract

Behavioral and neural evidence reveal a prospective goal-directed decision process that relies on mental simulation of the environment, and a retrospective habitual process that caches returns previously garnered from available choices. Artificial systems combine the two by simulating the environment up to some depth and then exploiting habitual values as proxies for consequences that may arise in the further future. Using a three-step task, we provide evidence that human subjects use such a normative plan-until-habit strategy, implying a spectrum of approaches that interpolates between habitual and goal-directed responding. We found that increasing time pressure led to shallower goal-directed planning, suggesting that a speed-accuracy tradeoff controls the depth of planning with deeper search leading to more accurate evaluation, at the cost of slower decision-making. We conclude that subjects integrate habit-based cached values directly into goal-directed evaluations in a normative manner.

Type: Article
Title: Adaptive integration of habits into depth-limited planning defines a habitual-goal-directed spectrum
Location: United States
Open access status: An open access version is available from UCL Discovery
DOI: 10.1073/pnas.1609094113
Publisher version: http://dx.doi.org/10.1073/pnas.1609094113
Additional information: Copyright © 2016 National Academy of Sciences.
Keywords: Habit, planning, reinforcement learning, speed/accuracy tradeoff, tree-based evaluation
UCL classification: UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology > Imaging Neuroscience
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neurosci Unit
URI: https://discovery.ucl.ac.uk/id/eprint/1528542
Downloads since deposit
95Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item