Galashov, A; Titsias, MK; György, A; Lyle, C; Pascanu, R; Teh, YW; Sahani, M; (2024) Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset. In: Proceedings of the Advances in Neural Information Processing Systems 37 (NeurIPS 2024). NeurIPS

Preview

Text NeurIPS-2024-non-stationary-learning-of-neural-networks-with-automatic-soft-parameter-reset-Paper-Conference.pdf - Published Version Download (4MB) | Preview

Abstract

Neural networks are most often trained under the assumption that data come from a stationary distribution. However, settings in which this assumption is violated are of increasing importance; examples include supervised learning with distributional shifts, reinforcement learning, continual learning and non-stationary contextual bandits. Here, we introduce a novel learning approach that automatically models and adapts to non-stationarity by linking parameters through an Ornstein-Uhlenbeck process with an adaptive drift parameter. The adaptive drift draws the parameters towards the distribution used at initialisation, so the approach can be understood as a form of soft parameter reset. We show empirically that our approach performs well in non-stationary supervised, and off-policy reinforcement learning settings.

Download activity - last month

Export as XMLCSVJSON

Download activity - last 12 months

Export as XMLCSVJSON

Downloads by country - last 12 months

Export as XMLCSVJSON

Archive Staff Only

View Item