Paul, A;
Sajid, N;
Gopalkrishnan, M;
Razi, A;
(2022)
Active Inference for Stochastic Control.
In:
Communications in Computer and Information Science.
(pp. pp. 669-680).
Springer International Publishing: Cham, Switzerland.
Preview |
Text
Sajid_2108.12245v1.pdf - Accepted Version Download (236kB) | Preview |
Abstract
Active inference has emerged as an alternative approach to control problems given its intuitive (probabilistic) formalism. However, despite its theoretical utility, computational implementations have largely been restricted to low-dimensional, deterministic settings. This paper highlights that this is a consequence of the inability to adequately model stochastic transition dynamics, particularly when an extensive policy (i.e., action trajectory) space must be evaluated during planning. Fortunately, recent advancements propose a modified planning algorithm for finite temporal horizons. We build upon this work to assess the utility of active inference for a stochastic control setting. For this, we simulate the classic windy grid-world task with additional complexities, namely: 1) environment stochasticity; 2) learning of transition dynamics; and 3) partial observability. Our results demonstrate the advantage of using active inference, compared to reinforcement learning, in both deterministic and stochastic settings.
Archive Staff Only
 |
View Item |
