UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Choose Your Path Wisely: Gradient Descent in a Bregman Distance Framework

Benning, M; Betcke, MM; Ehrhardt, MJ; Schönlieb, C-B; (2021) Choose Your Path Wisely: Gradient Descent in a Bregman Distance Framework. SIAM Journal on Imaging Sciences , 14 (2) pp. 814-843. 10.1137/20M1357500. Green open access

[thumbnail of Betcke_20m1357500.pdf]
Preview
Text
Betcke_20m1357500.pdf - Published Version

Download (3MB) | Preview

Abstract

We propose an extension of a special form of gradient descent---in the literature known as linearized Bregman iteration---to a larger class of nonconvex functions. We replace the classical (squared) two norm metric in the gradient descent setting with a generalized Bregman distance, based on a proper, convex, and lower semicontinuous function. The algorithm's global convergence is proven for functions that satisfy the Kurdyka--Łojasiewicz property. Examples illustrate that features of different scale are being introduced throughout the iteration, transitioning from coarse to fine. This coarse-to-fine approach with respect to scale allows us to recover solutions of nonconvex optimization problems that are superior to those obtained with conventional gradient descent, or even projected and proximal gradient descent. The effectiveness of the linearized Bregman iteration in combination with early stopping is illustrated for the applications of parallel magnetic resonance imaging, blind deconvolution, as well as image classification with neural networks.

Type: Article
Title: Choose Your Path Wisely: Gradient Descent in a Bregman Distance Framework
Open access status: An open access version is available from UCL Discovery
DOI: 10.1137/20M1357500
Publisher version: https://doi.org/10.1137/20M1357500
Language: English
Additional information: This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: nonconvex optimization, nonsmooth optimization, gradient descent, Bregman iteration, linearized Bregman iteration, parallel MRI, blind deconvolution, deep learning
UCL classification: UCL
UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10122178
Downloads since deposit
105Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item