Ritter, H;
Botev, A;
Barber, D;
(2018)
Online Structured Laplace Approximations for Overcoming Catastrophic Forgetting.
In: Bengio, S and Wallach, H and Larochelle, H and Grauman, K and CesaBianchi, N and Garnett, R, (eds.)
Proceedings of the 32nd Conference on Neural Information Processing Systems (NIPS 2018).
Neural Information Processing Systems (NIPS): Montréal, Canada.
Preview |
Text
osla.pdf - Published Version Download (520kB) | Preview |
Abstract
We introduce the Kronecker factored online Laplace approximation for overcoming catastrophic forgetting in neural networks. The method is grounded in a Bayesian online learning framework, where we recursively approximate the posterior after every task with a Gaussian, leading to a quadratic penalty on changes to the weights. The Laplace approximation requires calculating the Hessian around a mode, which is typically intractable for modern architectures. In order to make our method scalable, we leverage recent block-diagonal Kronecker factored approximations to the curvature. Our algorithm achieves over 90% test accuracy across a sequence of 50 instantiations of the permuted MNIST dataset, substantially outperforming related methods for overcoming catastrophic forgetting.
| Type: | Proceedings paper |
|---|---|
| Title: | Online Structured Laplace Approximations for Overcoming Catastrophic Forgetting |
| Event: | NIPS 2018 |
| Location: | Montreal, CANADA |
| Dates: | 02 December 2018 - 08 December 2018 |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | https://papers.nips.cc/book/advances-in-neural-inf... |
| Language: | English |
| Additional information: | This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | Science & Technology, Technology, Computer Science, Artificial Intelligence, Computer Science |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10076100 |
Loading...
Loading...Loading...Loading...Archive Staff Only
 |
View Item |
