FitzGerald, THB;
Penny, WD;
Bonnici, HM;
Adams, RA;
(2020)
Retrospective Inference as a Form of Bounded Rationality, and Its Beneficial Influence on Learning.
Frontiers in Artificial Intelligence
, 3
, Article 2. 10.3389/frai.2020.00002.
Preview |
Text
frai-03-00002.pdf - Published Version Download (1MB) | Preview |
Abstract
Probabilistic models of cognition typically assume that agents make inferences about current states by combining new sensory information with fixed beliefs about the past, an approach known as Bayesian filtering. This is computationally parsimonious, but, in general, leads to suboptimal beliefs about past states, since it ignores the fact that new observations typically contain information about the past as well as the present. This is disadvantageous both because knowledge of past states may be intrinsically valuable, and because it impairs learning about fixed or slowly changing parameters of the environment. For these reasons, in offline data analysis it is usual to infer on every set of states using the entire time series of observations, an approach known as (fixed-interval) Bayesian smoothing. Unfortunately, however, this is impractical for real agents, since it requires the maintenance and updating of beliefs about an ever-growing set of states. We propose an intermediate approach, finite retrospective inference (FRI), in which agents perform update beliefs about a limited number of past states (Formally, this represents online fixed-lag smoothing with a sliding window). This can be seen as a form of bounded rationality in which agents seek to optimize the accuracy of their beliefs subject to computational and other resource costs. We show through simulation that this approach has the capacity to significantly increase the accuracy of both inference and learning, using a simple variational scheme applied to both randomly generated Hidden Markov models (HMMs), and a specific application of the HMM, in the form of the widely used probabilistic reversal task. Our proposal thus constitutes a theoretical contribution to normative accounts of bounded rationality, which makes testable empirical predictions that can be explored in future work.
| Type: | Article |
|---|---|
| Title: | Retrospective Inference as a Form of Bounded Rationality, and Its Beneficial Influence on Learning |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.3389/frai.2020.00002 |
| Publisher version: | https://doi.org/10.3389/frai.2020.00002 |
| Language: | English |
| Additional information: | Copyright © 2020 FitzGerald, Penny, Bonnici and Adams. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) (http://creativecommons.org/licenses/by/4.0/). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
| Keywords: | bayesian inference, learning, cognition, retrospective inference, reversal learning, bounded rationality, hidden markov model |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10094952 |
Archive Staff Only
 |
View Item |
