Lyu, Zhaoyan;
Aminian, Gholamali;
Rodrigues, Miguel RD;
(2021)
Toward Minimal-Sufficiency in Regression Tasks: An Approach Based on a Variational Estimation Bottleneck.
In:
2021 IEEE 31ST INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING (MLSP).
IEEE: Gold Coast, Australia.
Preview |
Text
2021_MLSP_2021.pdf - Accepted Version Download (8MB) | Preview |
Abstract
We propose a new variational estimation bottleneck based on a mean-squared error metric to aid regression tasks. In particular, this bottleneck - which draws inspiration from a variational information bottleneck for classification counterparts - consists of two components: (1) one captures the notion of Vr -sufficiency that quantifies the ability for an estimator in some class of estimators Vr to infer the quantity of interest; (2) the other component appears to capture a notion of Vr - minimality that quantifies the ability of the estimator to generalize to new data. We demonstrate how to train this bottleneck for regression problems. We also conduct various experiments in image denoising and deraining applications showcasing that our proposed approach can lead to neural network regressors offering better performance without suffering from overfitting.
| Type: | Proceedings paper |
|---|---|
| Title: | Toward Minimal-Sufficiency in Regression Tasks: An Approach Based on a Variational Estimation Bottleneck |
| Event: | IEEE 31st International Workshop on Machine Learning for Signal Processing (MLSP) |
| Location: | ELECTR NETWORK |
| Dates: | 25 Oct 2021 - 28 Oct 2021 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/MLSP52302.2021.9596368 |
| Publisher version: | https://doi.org/10.1109/MLSP52302.2021.9596368 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | Deep learning, Information Bottleneck, regression, information theory, NETWORK |
| UCL classification: | UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Electronic and Electrical Eng UCL > Provost and Vice Provost Offices > UCL BEAMS UCL |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10150912 |
Loading...
Loading...Loading...Loading...Archive Staff Only
 |
View Item |
