UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

The Advantage of Conditional Meta-Learning for Biased Regularization and Fine-Tuning

Denevi, G; Pontil, M; Ciliberto, C; (2020) The Advantage of Conditional Meta-Learning for Biased Regularization and Fine-Tuning. In: Proceedings of NeurIPS 2020: Thirty-fourth Conference on Neural Information Processing Systems. Neural Information Processing Systems: Virtual conference. Green open access

[thumbnail of NeurIPS-2020-the-advantage-of-conditional-meta-learning-for-biased-regularization-and-fine-tuning-Paper.pdf]
Preview
Text
NeurIPS-2020-the-advantage-of-conditional-meta-learning-for-biased-regularization-and-fine-tuning-Paper.pdf - Published version

Download (1MB) | Preview

Abstract

Biased regularization and fine tuning are two recent meta-learning approaches. They have been shown to be effective to tackle distributions of tasks, in which the tasks’ target vectors are all close to a common meta-parameter vector. However, these methods may perform poorly on heterogeneous environments of tasks, where the complexity of the tasks’ distribution cannot be captured by a single meta- parameter vector. We address this limitation by conditional meta-learning, inferring a conditioning function mapping task’s side information into a meta-parameter vector that is appropriate for that task at hand. We characterize properties of the environment under which the conditional approach brings a substantial advantage over standard meta-learning and we highlight examples of environments, such as those with multiple clusters, satisfying these properties. We then propose a convex meta-algorithm providing a comparable advantage also in practice. Numerical experiments confirm our theoretical findings.

Type: Proceedings paper
Title: The Advantage of Conditional Meta-Learning for Biased Regularization and Fine-Tuning
Event: NeurIPS 2020: Thirty-fourth Conference on Neural Information Processing Systems
Open access status: An open access version is available from UCL Discovery
Publisher version: https://proceedings.neurips.cc/paper/2020/hash/0a7...
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10115253
Downloads since deposit
91Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item