Cullen, M;
Davey, B;
Friston, KJ;
Moran, RJ;
(2018)
Active Inference in OpenAI Gym: A Paradigm for Computational Investigations Into Psychiatric Illness.
Biological Psychiatry: Cognitive Neuroscience and Neuroimaging
, 3
(9)
pp. 809-818.
10.1016/j.bpsc.2018.06.010.
Preview |
Text
Friston_Active Inference on OpenAI Gym.pdf - Accepted Version Download (804kB) | Preview |
Abstract
Background: Artificial intelligence has recently attained humanlike performance in a number of gamelike domains. These advances have been spurred by brain-inspired architectures and algorithms such as hierarchical filtering and reinforcement learning. OpenAI Gym is an open-source platform in which to train, test, and benchmark algorithms—it provides a range of tasks, including those of classic arcade games such as Doom. Here we describe how the platform might be used as a simulation, test, and diagnostic paradigm for psychiatric conditions. Methods: To illustrate how active inference models of game play could be used to test mechanistic and algorithmic properties of psychiatric disorders, we provide two exemplar analyses. The first speaks to the impact of aging on cognition, examining game-play behaviors in a model of aging in which we compared age-dependent changes of younger (n = 9, 22 ± 1 years of age) and older (n = 7, 56 ± 5 years of age) adult players. The second is an illustration of a putative feature of anhedonia in which we simulated diminished sensitivity to reward. Results: These simulations demonstrate how active inference can be used to test predicted changes in both neurobiology and beliefs in psychiatric cohorts. We show that, as well as behavioral measures, putative neural correlates of active inference can be simulated, and hypothesized (model-based) differences in local field potentials and blood oxygen level–dependent responses can be produced. Conclusions: We show that active inference, through epistemic and value-based goals, enables simulated subjects to actively develop detailed representations of gaming environments, and we demonstrate the use of a principled algorithmic and neurobiological framework for testing hypotheses in psychiatric illness.
| Type: | Article |
|---|---|
| Title: | Active Inference in OpenAI Gym: A Paradigm for Computational Investigations Into Psychiatric Illness |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1016/j.bpsc.2018.06.010 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | Active inference, Computational phenotyping, Com putational psychiatry, Free energy principle,Game-based imaging biomarkers, Markov decision process |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology > Imaging Neuroscience |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10056745 |
Archive Staff Only
 |
View Item |
