Amjad, Jaweria;
Lyu, Zhaoyan;
Rodrigues, Miguel RD;
(2021)
Regression with Deep Neural Networks: Generalization Error Guarantees, Learning Algorithms, and Regularizers.
In:
2021 29th European Signal Processing Conference (EUSIPCO).
(pp. pp. 1481-1485).
IEEE: Dublin, Ireland.
Preview |
Text
amjad2020generalizationEUSIPCO.pdf - Accepted Version Download (1MB) | Preview |
Abstract
We present new data-dependent characterizations of the generalization capability of deep neural networks based data representations within the context of regression tasks. In particular, we propose new generalization error bounds that depend on various elements associated with the learning problem such as the complexity of the data space, the cardinality of the training set, and the input-output Jacobian of the deep neural network. Moreover, building upon our bounds, we propose new regularization strategies constraining the network Lipschitz properties through norms of the network gradient. Experimental results show that our newly proposed regularization techniques can deliver state-of-the-art performance in comparison to established weight-based regularization.
| Type: | Proceedings paper |
|---|---|
| Title: | Regression with Deep Neural Networks: Generalization Error Guarantees, Learning Algorithms, and Regularizers |
| Event: | 2021 29th European Signal Processing Conference (EUSIPCO) |
| Location: | ELECTR NETWORK |
| Dates: | 23 Aug 2021 - 27 Aug 2021 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.23919/EUSIPCO54536.2021.9616069 |
| Publisher version: | https://doi.org/10.23919/EUSIPCO54536.2021.9616069 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | Deep learning, Training, Jacobian matrices, Buildings, Signal processing algorithms, Europe, Signal processing |
| UCL classification: | UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Electronic and Electrical Eng UCL > Provost and Vice Provost Offices > UCL BEAMS UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neurosci Unit UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10151169 |
Archive Staff Only
 |
View Item |
