UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Learning To Learn Around A Common Mean

Denevi, G; Ciliberto, C; Stamos, D; Pontil, M; (2018) Learning To Learn Around A Common Mean. In: Bengio, S and Wallach, H and Larochelle, H and Grauman, K and CesaBianchi, N and Garnett, R, (eds.) Advances in Neural Information Processing Systems 31. NIPS Proceedings: Montréal, Canada. Green open access

[thumbnail of 8220-learning-to-learn-around-a-common-mean.pdf]
Preview
Text
8220-learning-to-learn-around-a-common-mean.pdf - Published version

Download (863kB) | Preview

Abstract

The problem of learning-to-learn (LTL) or meta-learning is gaining increasing attention due to recent empirical evidence of its effectiveness in applications. The goal addressed in LTL is to select an algorithm that works well on tasks sampled from a meta-distribution. In this work, we consider the family of algorithms given by a variant of Ridge Regression, in which the regularizer is the square distance to an unknown mean vector. We show that, in this setting, the LTL problem can be reformulated as a Least Squares (LS) problem and we exploit a novel meta- algorithm to efficiently solve it. At each iteration the meta-algorithm processes only one dataset. Specifically, it firstly estimates the stochastic LS objective function, by splitting this dataset into two subsets used to train and test the inner algorithm, respectively. Secondly, it performs a stochastic gradient step with the estimated value. Under specific assumptions, we present a bound for the generalization error of our meta-algorithm, which suggests the right splitting parameter to choose. When the hyper-parameters of the problem are fixed, this bound is consistent as the number of tasks grows, even if the sample size is kept constant. Preliminary experiments confirm our theoretical findings, highlighting the advantage of our approach, with respect to independent task learning.

Type: Proceedings paper
Title: Learning To Learn Around A Common Mean
Event: Neural Information Processing Systems 2018
Location: Montreal, CANADA
Dates: 02 December 2018 - 08 December 2018
Open access status: An open access version is available from UCL Discovery
Publisher version: https://papers.nips.cc/paper/8220-learning-to-lear...
Language: English
Additional information: This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10112123
Downloads since deposit
87Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item