UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Confidence modulates exploration and exploitation in value-based learning

Boldt, A; Blundell, C; De Martino, B; (2019) Confidence modulates exploration and exploitation in value-based learning. Neurosci Conscious , 2019 (1) , Article niz004. 10.1093/nc/niz004. Green open access

[thumbnail of niz004.pdf]
Preview
Text
niz004.pdf - Published Version

Download (584kB) | Preview

Abstract

Uncertainty is ubiquitous in cognitive processing. In this study, we aim to investigate the ability agents possess to track and report the noise inherent in their mental operations, often in the form of confidence judgments. Here, we argue that humans can use uncertainty inherent in their representations of value beliefs to arbitrate between exploration and exploitation. Such uncertainty is reflected in explicit confidence judgments. Using a novel variant of a multi-armed bandit paradigm, we studied how beliefs were formed and how uncertainty in the encoding of these value beliefs (belief confidence) evolved over time. We found that people used uncertainty to arbitrate between exploration and exploitation, reflected in a higher tendency toward exploration when their confidence in their value representations was low. We furthermore found that value uncertainty can be linked to frameworks of metacognition in decision making in two ways. First, belief confidence drives decision confidence, i.e. people's evaluation of their own choices. Second, individuals with higher metacognitive insight into their choices were also better at tracing the uncertainty in their environment. Together, these findings argue that such uncertainty representations play a key role in the context of cognitive control.

Type: Article
Title: Confidence modulates exploration and exploitation in value-based learning
Location: England
Open access status: An open access version is available from UCL Discovery
DOI: 10.1093/nc/niz004
Publisher version: https://doi.org/10.1093/nc/niz004
Language: English
Additional information: Copyright © The Author(s) 2019. Published by Oxford University Press. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.
Keywords: confidence, exploration–exploitation dilemma, metacognition, uncertainty, value-based choice
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences > Institute of Cognitive Neuroscience
URI: https://discovery.ucl.ac.uk/id/eprint/10075023
Downloads since deposit
85Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item