UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Lautum Regularization for Semi-Supervised Transfer Learning

Jakubovitz, D; Giryes, R; Rodrigues, MRD; (2020) Lautum Regularization for Semi-Supervised Transfer Learning. In: Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). (pp. pp. 763-767). IEEE: Seoul, Korea (South). Green open access

[img]
Preview
Text
Rodrigues_Jakubovitz_Lautum_Regularization_for_Semi-Supervised_Transfer_Learning_ICCVW_2019_paper.pdf - Accepted version

Download (245kB) | Preview

Abstract

Transfer learning is a very important tool in deep learning as it allows propagating information from one "source dataset" to another "target dataset", especially in the case of a small number of training examples in the latter. Yet, discrepancies between the underlying distributions of the source and target data are commonplace and are known to have a substantial impact on algorithm performance. In this work we suggest a novel information theoretic approach for the analysis of the performance of deep neural networks in the context of transfer learning. We focus on the task of semi-supervised transfer learning, in which unlabeled samples from the target dataset are available during the network training on the source dataset. Our theory suggests that one may improve the transferability of a deep neural network by imposing a Lautum information based regularization that relates the network weights to the target data. We demonstrate the effectiveness of the proposed approach in various transfer learning experiments.

Type: Proceedings paper
Title: Lautum Regularization for Semi-Supervised Transfer Learning
Event: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
Location: Seoul, SOUTH KOREA
Dates: 27 October 2019 - 02 November 2019
Open access status: An open access version is available from UCL Discovery
DOI: 10.1109/ICCVW.2019.00100
Publisher version: https://doi.org/10.1109/ICCVW.2019.00100
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Transfer Learning, Semi Supervised Learning, Information Theory
UCL classification: UCL
UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Electronic and Electrical Eng
URI: https://discovery.ucl.ac.uk/id/eprint/10109748
Downloads since deposit
13Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item