UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Regression with Deep Neural Networks: Generalization Error Guarantees, Learning Algorithms, and Regularizers

Amjad, Jaweria; Lyu, Zhaoyan; Rodrigues, Miguel RD; (2021) Regression with Deep Neural Networks: Generalization Error Guarantees, Learning Algorithms, and Regularizers. In: 2021 29th European Signal Processing Conference (EUSIPCO). (pp. pp. 1481-1485). IEEE: Dublin, Ireland. Green open access

[thumbnail of amjad2020generalizationEUSIPCO.pdf]
Preview
Text
amjad2020generalizationEUSIPCO.pdf - Accepted Version

Download (1MB) | Preview

Abstract

We present new data-dependent characterizations of the generalization capability of deep neural networks based data representations within the context of regression tasks. In particular, we propose new generalization error bounds that depend on various elements associated with the learning problem such as the complexity of the data space, the cardinality of the training set, and the input-output Jacobian of the deep neural network. Moreover, building upon our bounds, we propose new regularization strategies constraining the network Lipschitz properties through norms of the network gradient. Experimental results show that our newly proposed regularization techniques can deliver state-of-the-art performance in comparison to established weight-based regularization.

Type: Proceedings paper
Title: Regression with Deep Neural Networks: Generalization Error Guarantees, Learning Algorithms, and Regularizers
Event: 2021 29th European Signal Processing Conference (EUSIPCO)
Location: ELECTR NETWORK
Dates: 23 Aug 2021 - 27 Aug 2021
Open access status: An open access version is available from UCL Discovery
DOI: 10.23919/EUSIPCO54536.2021.9616069
Publisher version: https://doi.org/10.23919/EUSIPCO54536.2021.9616069
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Deep learning, Training, Jacobian matrices, Buildings, Signal processing algorithms, Europe, Signal processing
UCL classification: UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Electronic and Electrical Eng
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neurosci Unit
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
URI: https://discovery.ucl.ac.uk/id/eprint/10151169
Downloads since deposit
125Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item