Chen, AG;
Benrimoh, D;
Parr, T;
Friston, KJ;
(2020)
A Bayesian Account of Generalist and Specialist Formation Under the Active Inference Framework.
Frontiers in Artificial Intelligence
, 3
, Article 69. 10.3389/frai.2020.00069.
Preview |
Text
frai-03-00069.pdf - Published Version Download (1MB) | Preview |
Abstract
This paper offers a formal account of policy learning, or habitual behavioral optimization, under the framework of Active Inference. In this setting, habit formation becomes an autodidactic, experience-dependent process, based upon what the agent sees itself doing. We focus on the effect of environmental volatility on habit formation by simulating artificial agents operating in a partially observable Markov decision process. Specifically, we used a "two-step" maze paradigm, in which the agent has to decide whether to go left or right to secure a reward. We observe that in volatile environments with numerous reward locations, the agents learn to adopt a generalist strategy, never forming a strong habitual behavior for any preferred maze direction. Conversely, in conservative or static environments, agents adopt a specialist strategy; forming strong preferences for policies that result in approach to a small number of previously-observed reward locations. The pros and cons of the two strategies are tested and discussed. In general, specialization offers greater benefits, but only when contingencies are conserved over time. We consider the implications of this formal (Active Inference) account of policy learning for understanding the relationship between specialization and habit formation.
| Type: | Article |
|---|---|
| Title: | A Bayesian Account of Generalist and Specialist Formation Under the Active Inference Framework |
| Location: | Switzerland |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.3389/frai.2020.00069 |
| Publisher version: | http://dx.doi.org/10.3389/frai.2020.00069 |
| Language: | English |
| Additional information: | © 2020 Chen, Benrimoh, Parr and Friston. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
| Keywords: | Bayesian, active inference, generative model, learning strategies, predictive processing, preferences |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology > Imaging Neuroscience |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10125220 |
Archive Staff Only
 |
View Item |
