UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Nesterov's accelerated gradient and momentum as approximations to regularised update descent

Botev, A; Lever, G; Barber, D; (2017) Nesterov's accelerated gradient and momentum as approximations to regularised update descent. In: Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN). IEEE: Anchorage, AK, USA. Green open access

[thumbnail of 07966082.pdf]
Preview
Text
07966082.pdf - Published Version

Download (206kB) | Preview

Abstract

We present a unifying framework for adapting the update direction in gradient-based iterative optimization methods. As natural special cases we re-derive classical momentum and Nesterov's accelerated gradient method, lending a new intuitive interpretation to the latter algorithm. We show that a new algorithm, which we term Regularised Gradient Descent, can converge more quickly than either Nesterov's algorithm or the classical momentum algorithm.

Type: Proceedings paper
Title: Nesterov's accelerated gradient and momentum as approximations to regularised update descent
Event: 2017 International Joint Conference on Neural Networks (IJCNN)
Open access status: An open access version is available from UCL Discovery
DOI: 10.1109/IJCNN.2017.7966082
Publisher version: https://doi.org/10.1109/IJCNN.2017.7966082
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10062712
Downloads since deposit
2,673Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item