UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Compositional generalization in multi-armed bandits

Saanum, T; Eric, S; Speekenbrink, M; (2021) Compositional generalization in multi-armed bandits. In: Proceedings of the Annual Meeting of the Cognitive Science Society. Cognitive Science Society Green open access

[thumbnail of qt5nn9q6zc.pdf]
Preview
Text
qt5nn9q6zc.pdf - Published version

Download (1MB) | Preview

Abstract

To what extent do human reward learning and decision-making rely on the ability to represent and generate richly structured relationships between options? We provide evidence that structure learning and the principle of compositionality play crucial roles in human reinforcement learning. In a new multi-armed bandit paradigm, we found evidence that participants are able to learn representations of different reward structures and combine them to make correct generalizations about options in novel contexts. Moreover, we found substantial evidence that participants transferred knowledge of simpler reward structures to make compositional generalizations about rewards in complex contexts. This allowed participants to accumulate more rewards earlier, and to explore less whenever such knowledge transfer was possible. We also provide a computational model which is able to generalize and compose knowledge for complex reward structures. This model describes participant behaviour in the compositional generalization task better than various other models of decision-making and transfer learning.

Type: Proceedings paper
Title: Compositional generalization in multi-armed bandits
Event: 43rd Annual Meeting of the Cognitive Science Society
Open access status: An open access version is available from UCL Discovery
Publisher version: https://escholarship.org/uc/item/5nn9q6zc
Language: English
Additional information: This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Compositionality; Reinforcement learning; Trans-fer learning; Gaussian Processes
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences > Experimental Psychology
URI: https://discovery.ucl.ac.uk/id/eprint/10138387
Downloads since deposit
4Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item