UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Bilevel Programming for Hyperparameter Optimization and Meta-Learning

Franceschi, L; Frasconi, P; Salzo, S; Grazzi, R; Pontil, M; (2018) Bilevel Programming for Hyperparameter Optimization and Meta-Learning. In: Dy, JG and Krause, A, (eds.) Proceedings of the 25th International Conference on Machine Learning (2018). (pp. pp. 1563-1572). PMLR (Proceedings of Machine Learning Research): Stockholm. Green open access

[thumbnail of franceschi18a.pdf]
Preview
Text
franceschi18a.pdf - Published version

Download (1MB) | Preview

Abstract

We introduce a framework based on bilevel programming that unifies gradient-based hyperparameter optimization and meta-learning. We show that an approximate version of the bilevel problem can be solved by taking into explicit account the optimization dynamics for the inner objective. Depending on the specific setting, the outer variables take either the meaning of hyperparameters in a supervised learning problem or parameters of a meta-learner. We provide sufficient conditions under which solutions of the approximate problem converge to those of the exact problem. We instantiate our approach for meta-learning in the case of deep learning where representation layers are treated as hyperparameters shared across a set of training episodes. In experiments, we confirm our theoretical findings, present encouraging results for few-shot learning and contrast the bilevel approach against classical approaches for learning-to-learn.

Type: Proceedings paper
Title: Bilevel Programming for Hyperparameter Optimization and Meta-Learning
Event: 25th International Conference on Machine Learning, 10-15 July 2018, Stockhom. Sweden
Open access status: An open access version is available from UCL Discovery
Publisher version: http://proceedings.mlr.press/v80/
Language: English
Additional information: This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10073436
Downloads since deposit
16Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item