UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Toward Minimal-Sufficiency in Regression Tasks: An Approach Based on a Variational Estimation Bottleneck

Lyu, Zhaoyan; Aminian, Gholamali; Rodrigues, Miguel RD; (2021) Toward Minimal-Sufficiency in Regression Tasks: An Approach Based on a Variational Estimation Bottleneck. In: 2021 IEEE 31ST INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING (MLSP). IEEE: Gold Coast, Australia. Green open access

[thumbnail of 2021_MLSP_2021.pdf]
Preview
Text
2021_MLSP_2021.pdf - Accepted Version

Download (8MB) | Preview

Abstract

We propose a new variational estimation bottleneck based on a mean-squared error metric to aid regression tasks. In particular, this bottleneck - which draws inspiration from a variational information bottleneck for classification counterparts - consists of two components: (1) one captures the notion of Vr -sufficiency that quantifies the ability for an estimator in some class of estimators Vr to infer the quantity of interest; (2) the other component appears to capture a notion of Vr - minimality that quantifies the ability of the estimator to generalize to new data. We demonstrate how to train this bottleneck for regression problems. We also conduct various experiments in image denoising and deraining applications showcasing that our proposed approach can lead to neural network regressors offering better performance without suffering from overfitting.

Type: Proceedings paper
Title: Toward Minimal-Sufficiency in Regression Tasks: An Approach Based on a Variational Estimation Bottleneck
Event: IEEE 31st International Workshop on Machine Learning for Signal Processing (MLSP)
Location: ELECTR NETWORK
Dates: 25 Oct 2021 - 28 Oct 2021
Open access status: An open access version is available from UCL Discovery
DOI: 10.1109/MLSP52302.2021.9596368
Publisher version: https://doi.org/10.1109/MLSP52302.2021.9596368
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Deep learning, Information Bottleneck, regression, information theory, NETWORK
UCL classification: UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Electronic and Electrical Eng
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL
URI: https://discovery.ucl.ac.uk/id/eprint/10150912
Downloads since deposit
43Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item